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CHAPTER  I 


INTRODUCTION 

Overview 

This  research  effort  is  intended  to  be  an  initial 
examination  of  contract  modification  instruments.  A scant 
amount  of  information  is  available  on  the  procedures  used 
to  modify  contracts.  It  is  anticipated  that  the  results  of 
this  effort  can  be  used  as  a platform  for  further  research 
into  this  complex  area. 

The  thesis  is  presented  in  four  chapters.  Chapter 
I is  the  introduction,  Chapter  II  describes  the  methodology 
used,  Chapter  III  analyzes  the  data,  and  Chapter  IV  presents 
the  research  findings.  Chapter  I contains  a statement  of 
the  problem,  background  on  the  problem,  objective  of  the 
study,  and  the  research  questions.  Chapter  II  provides 
information  on  the  questionnaire,  population  and  sample, 
data  analysis  used,  and  a summary  of  assumptions  and  limita- 
tions. Chapter  III  contains  a discussion  on  the  computer 
technology  and  statistical  methods  used  along  with  the 
analysis  of  the  data.  Chapter  IV  presents  the  answers  to 
the  research  question,  other  pertinent  findings,  and  recom- 
mendations for  further  research.  Chapter  I will  begin  with 
a statement  of  the  problem. 
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statement  of  the  Problem 


One  major  problem  facing  the  Department  of  Defense 
(DOD)  is  that  of  cost  growth  (3:22).  Cost  growth  is  defined 
as  the  difference  between  the  initial  cost  and  the 
final  cost  of  the  contract  (9:7).  The  problem  of  cost 
growth  is  particularly  crucial  today  in  the  face  of  infla- 
tion and  continual  Congressional  scrutiny  of  the  procurement 
process  (3: 166) . 

One  significant  contributor  to  cost  growth  is  con- 
tract modifications  (3:23).  This  is  evidenced  by  DOD 
figures  relative  to  contract  modifications  for  the  five- 
year  period  1966-1970.  During  this  time,  over  60,000  modi- 
fications were  accomplished  at  a cost  in  excess  of  $9.4 
billion  (6:  364)  . 

Looking  at  modification  costs  as  a percentage  of 
the  original  cost  estimate  also  points  out  the  impact  of 
change  on  cost  growth.  A Logistics  Management  Institute 
report  dated  January,  1964,  identifies  $7.6  billion  in  cost 
growth  on  139  weapon  system  contracts  originally  valued  at 
$15.3  billion.  This  represents  an  increase  of  almost  50 
percent.  A 1971  General  Accounting  Office  (GAO)  report 
also  relates  similar  findings  (6:364-365). 

Since  contract  modifications  are  alterations  in 
the  original  contract,  action  taken  to  make  the  alteration 
would  imply  an  additional  associated  cost  in  administrative 
burden  as  well  as  a potential  increase  in  the  price  of  the 
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contract.  A formal  change  order  may  involve  more  than  one 
contract  change;  therefore,  the  negotiation  and  control 
problems  involved  may  be  even  more  burdensome  than  it 
appears  (6:364) . 

In  1969,  for  example,  the  Defense  Department  buy- 
ing agency  with  the  fewest  Change  Orders  (the  Army 
Material  Command)  approved  more  than  19,000  engineer- 
ing changes.  This  represents  an  approval  rate  of  more 
than  eight  per  hour  [6:364]. 

Fox  indicates  that  contract  changes  do  not  account 
for  all  of  management's  problems,  but  the  large  numbers  of 
contract  changes  tend  to  reduce  any  Government  capability 
to  control  contract  program  costs,  schedules,  and  perform- 
ance. He  further  states  that  Government  contract  manage- 
ment offices  do  not  have  the  necessary  number  of  qualified 
personnel  to  review  and  evaluate  even  the  major  changes 
properly  (6:381) . 

The  reasons  for  modifying  contracts  are  many,  and 
range  from  quantity  and  schedule  changes  to  engineering  and 
technical  changes  (8:6). 

The  need  for  change  control  should  be  obvious. 

Most  changes  increase  costs  and/or  delay  delivery  of 
the  product.  Uncontrolled  changes  often  lead  to  dif- 
ferent product  configurations  that  seriously  compli- 
cate the  operation  and  support  of  the  equipment  in  the 
field.  Poorly  timed  changes  may  disrupt  contractor 
operations  unnecessarily,  create  scrap  or  rework,  and 
generally  increase  cost  to  the  Government.  Delayed 
changes,  on  the  other  hand,  may  jeopardize  corrective 
actions  designed  to  avoid  fatal  or  serious  injury  to 
operating  personnel  [l:XI-30]. 

The  problem  facing  DOD  procurement  agencies  is  to 
effectively  manage  contract  modifications  while  concurrently 
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curbing  costs  and  making  the  modification  process  as  effi- 
cient as  possible.  The  contract  modification  instruments 
currently  being  used  by  DOD  contract  placement  and  manage- 
ment agencies  to  implement  contract  changes  need  to  be 
analyzed.  The  analysis  can  be  used  to  determine  which 
contract  modification  instruments  are  preferred  by  the  dif- 
ferent agencies;  and  to  identify  the  most  flexible,  manage- 
able, cost  effective,  and  timely  instriunents  for  modifying 
contracts . 


Background 

This  section  develops  operating  definitions  that 
apply  to  the  technical  procurement  terms  used.  A justifi- 
cation for  the  research  is  also  included.  The  last  part  of 
the  section  discusses  the  limitations  of  the  research  effort. 


Operating  Definitions 

Government  contracting  is  considered  to  be  a highly 
technical  business  (3:1).  Some  definitions  were  developed 
to  aid  in  understanding  the  immediate  terms  under  discus- 
sion. 


Contract  modification  instrument.  A contract  modi- 
fication as  defined  by  the  Armed  Services  Procurement  Regu- 
lation (ASPR) , 


. . . means  any  written  alteration  in  the  specifi- 
cation, delivery  point,  rate  of  delivery,  contract 
period,  price,  quantity,  or  other  contract  provisions 
of  an  existing  contract,  whether  accomplished  by 


unilateral  action  in  accordance  with  a contract  pro- 
vision, or  by  mutual  action  of  the  parties  to  the  con- 
tract. It  includes  (i)  bilateral  actions  such  as  sup- 
plemental agreements,  and  (ii)  unilateral  actions  such 
as  change  orders,  orders  for  provisioned  items, 
administrative  changes,  notices  of  termination  and 
notices  of  the  exercise  of  a contract  option  [17:1:15].^ 

A contract  modification  instrument  is  defined  here 
as  a procedure  used  to  accomplish  contract  modifications. 

A contract  modification  instrument  is  different  than  a 
Standard  Form  30,  Amendment  of  Solicitation/Modification  of 
Contract  (17:26:2),  which  is  a specific  form  that  is  used 
to  formalize  all  contract  modification  instruments;  con- 
versely, a contract  modification  instrument  is  a procedure 
for  modifying  a contract.  For  purposes  of  this  research, 
contract  modification  instruments  were  categorized  into  the 
following  six  general  types: 

1.  unilateral  change  order  issued  under  authority 
of  the  Changes  clause; 

2.  bilateral  supplemental  agreement  issued  under 
authority  of  the  Changes  clause; 

3.  unilateral  change  order  issued  under  authority 
of  a contract  provision  other  than  the  Changes  clause; 

4.  bilateral  supplemental  agreement  issued  under 
authority  of  a contract  provision  other  than  the  Changes 
clause; 

^Pages  in  the  Armed  Services  Procurement  Regulation 
are  numbered  within  each  section.  As  constructed,  this 
cite  indicates  the  17th  reference,  section  1,  and  pa*ge  15. 
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5.  bilateral  supplemental  agreement  for  perform- 
ance outside  the  scope  of  the  contract;  and 

6.  constructive  change  order  based  on  some  informal 
action  on  the  part  of  the  Government  that  implies  a con- 
tract is  to  be  modified. 

A unilateral  contract  modification  is  a one-sided 
action  by  the  Government,  while  a bilateral  contract  modi- 
fication " . . .is  agreed  to  and  executed  by  both  the 

V 

Government  and  the  contractor  [2:135]." 

Unilateral  change  order  issued  under  authority  of 
the  Changes  clause.  Change  orders  are  generally  thought 
of  as  being  authorized  only  by  the  Changes  clause.  In  fact, 
one  definition  in  ASPR  states  exactly  that. 

Change  Order  means  a written  order  signed  by  the 
contracting  officer,  directing  the  contractor  to  make 
changes  which  the  Changes  clause  of  the  contract 
authorized  the  contracting  officer  to  order  without 
the  consent  of  the  contractor  [17:1:15]. 

Fox  indicates  that  the  change  order  is  a unilateral  action 

that  implies  the  contracting  officer's  intent  to  enter  a 

supplemental  agreement  (6:361).  In  Section  XXVI  of  ASPR, 

change  orders  are  further  defined  as  being  ".  . . pursuant 

to  other  clauses  of  the  contract  invoking  the  Changes 

clause  procedures  [17:26:3]."  This  research  considers  a 

change  order  as  a contract  modification  issued  under  the 

authority  of  the  Changes  clause,  or  issued  under  the 

authority  of  any  other  clause  in  the  contract  using  the 
Changes  clause  procedures. 
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There  are  twelve  different  Changes  clauses  con- 
tained in  ASPR.  The  Changes  clause  used  in  fixed-price 
type  supply  contracts  allows  the  Government  to  make  uni- 
lateral changes  in; 

. . . (i)  drawings,  designs,  or  specifications, 

where  the  supplies  to  be  furnished  are  to  be  spe- 
cifically manufactured  for  the  Government  in  accord- 
ance therewith;  (ii)  method  of  shipment  or  packing; 
and  (iii)  place  of  delivery  [17:7:3]. 

The  Changes  clauses  used  in  cost-reimbursement  type  supply 
contracts  (17:7:165),  fixed-price  research  and  development 
contracts  (17:7:240),  and  cost-reimbursement  type  research 
and  development  contracts  (17:7:265)  all  contain  essen- 
tially the  same  authority  as  the  fixed-price  type  supply 
contract  Changes  clause.  The  Changes  clause  used  in  time 
and  materials  and  labor  hours  contracts  includes  the  addi- 
tional authority  to  change  "...  the  amount  of  Government 
furnished  property  [17:7:375]." 

The  Changes  clause  required  for  use  in  fixed-price 
type  construction  contracts  provides  for; 

. . . any  changes  in  the  work  within  the  general 
scope  of  the  contract,  including  but  not  limited  to 
changes;  (i)  in  the  specifications  (including  drawings 
and  designs);  (ii)  in  the  method  or  manner  of  perform- 
ance of  the  work;  (iii)  in  the  Government- furnished 
facilities,  equipment,  materials,  services,  or  site; 
or  (iv)  directing  acceleration  in  the  performance  of 
the  work  [17:7:272]. 

The  other  Changes  clauses  related  to  the  fixed-price  type 

construction  contract  clause  are  the  cost-reimbursement 

type  construction  contracts  clause  (17:7:319),  the  fixed- 
price  type  architect-engineer  contracts  clause  (17:7:332), 
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and  the  facilities  contracts  (17:7:343)  and  the  facilities 
acquisition  contracts  (17:7:360)  clauses.  They  all  basic- 
ally provide  for  changes  within  the  general  scope  of  the 
contract. 

The  Changes  clause  used  in  stevedoring  contracts 
allows  the  Government  to  " . . . make  changes  within  the 
general  scope  of  this  contract  [17:7:399]."  In  addition 
to  this  statement,  the  Changes  clause  used  in  fixed  price 
type  services  contracts  includes  ".  . .in  the  definition 
of  services  to  be  performed,  and  the  terms  (i.e.,  hours  of 
the  day,  days  of  the  week,  etc.)  and  place  of  performance 
thereof  [17:7:443]."  The  Changes  clause  for  cost  reimburse- 
ment type  service  contracts  is  worded  the  same,  except  I 

i 

"definition  of  tasks"  is  included  (17:7:458).  ^ 

Although  Changes  clauses  may  be  worded  somewhat  | 

differently,  they  all  basically  contain  the  same  common  | 

! 

1 

elements.  j 

The  essential  (and  common)  elements  of  a Changes  ] 

clause  include  the  requirements  that  (1)  a change  must  i 

^ be  within  the  scope  of  the  contract,  (2)  the  order  to 

change  must  be  in  writing,  and  (3)  the  ordered  change  i 

must  be  at  the  order  of  an  authorized  Government 
officer  [2:136]. 

The  Government  contracting  officer  has  the  authority  to  ' 

I 

issue  a unilateral  change  order  to  a contract  based  on  the  j 

advance  agreement  between  the  Government  and  the  contractor 
to  include  the  Changes  clause  in  the  original  contract 
(2:136).  If  the  unilateral  change  order  is  issued  without 
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a prior  pricing  agreement  between  the  Government  and  the 
contractor,  the  contractor  may  be  entitled  to  an  equitable 
adjustment  in  the  terms  of  the  contract  (17:7:3).  In  addi- 
tion, an  equitable  adjustment  negotiated  after  a unilateral 
change  order  will  necessitate  a bilateral  supplemental 
agreement  definitizing  the  final  negotiated  agreement 
(19:1)  . 

Bilateral  supplemental  agreement  issued  under 
authority  of  the  Changes  clause.  A supplemental  agreement 
is  a bilateral  agreeitient.  A supplemental  agreement  does 
not  require  specific  consummation  authority  within  the 
bounds  of  the  contract.  As  defined  in  ASPR,  "supplemental 
agreement  means  any  contract  modification  which  is 
accomplished  by  the  mutual  action  of  the  parties  [17:1:18]." 

The  principal  use  of  the  supplemental  agreement  is 
to  bring  about  an  equitable  adjustment  between  the 
Government  and  the  contractor,  as  in  the  case  of  modi- 
fying contract  price  or  delivery  terms  as  a result  of 
the  effects  of  a change  order  [l:XI-6]. 

A bilateral  supplemental  agreement  can  also  be  issued  under 

the  authority  of  the  Changes  clause,  in  effect,  providing 

the  preferred  forward  pricing  of  t.;e  modification  (17:26:7). 

"If  an  equitable  adjustment  in  the  contract  price  or 

delivery  term.s  or  both  can  be  agreed  upon  in  advance,  only 

a supplemental  agreement  need  be  issued  . . . [17:26:4]." 

A contract  modification  to  a construction  contract  issued 

under  a Changes  clause  requires  a definite  statement  about 
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any  contract  price  change,  and  "the  contractor  should  be 
requested  to  indicate  his  acceptance  . . . [17:16:29]." 


Contract  provisions  other  than  the  Changes  clause. 
Contract  modifications  issued  under  authority  of  contract 
provisions  other  than  the  Changes  clause  can  be  either 
unilateral  change  orders  or  bilateral  supplemental  agree- 
ments, using  modification  procedures  similar  to  Changes 
clause  authorized  modifications.  The  third  and  fourth 
general  types  of  contract  modification  instruments  are  the 
unilateral  change  order  issued  under  authority  of  a con- 
tract provision  other  than  the  Changes  clause  (17:26:3), 
and  the  bilateral  supplemental  agreement  issued  under 
authority  of  a contract  provision  other  than  the  Changes 
clause  (17:26:2,16:29).  The  processing  of  contract  modi- 
fications issued  pursuant  to  other  clauses  of  the  contract 
invoking  the  Changes  clause  procedures  is  the  same  as  pro- 
cessing modifications  under  the  Changes  clause  (17:26:3). 

There  are  numerous  clauses  within  a contract  that 
invoke  the  Changes  clause  procedures.  A fixed-price  type 
supply  contract  contains  an  Inspection  clause  that  allows 
the  Government  to  reject  defective  parts  and  purchase 
acceptable  parts  elsewhere,  to  default  the  contractor,  or 
to  accept  the  defective  parts  and  pay  a reduced  price 
(17:7:4).  This  clause  is  similar  to  the  Inspection  of 
Supplies  and  Correction  of  Defects  clause  used  in 


iin 
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cost-reimbursement  type  supply  contracts  (17:7:174),  the 
Inspection  clause  for  use  in  fixed-price  type  research  and 
development  contracts  (17:7:201),  and  the  Inspection  and 
Acceptance  clause  for  use  in  construction  contracts  (17:7: 
277)  . 

The  Inspection  of  Services  Clause  for  use  in  ser- 
vice contracts  is  broader  in  content.  The  contractor  may 
be  required  to  reaccomplish  unacceptable  work.  If  imprac- 
tical to  reaccomplish  the  work,  the  Government  may  require 
the  contractor  to  take  steps  to  insure  it  will  not  happen 
again  and  reduce  the  contract  price  to  reflect  the  reduced 
services.  If  the  contractor  does  not  improve  his  perform- 
ance, the  Government  may  have  the  services  performed  and 
change  the  cost  to  the  contractor;  or  the  contract  may  be 
terminated  (17:7:443). 

The  Government  Property  clause  for  use  in  fixed 
price  contracts  is  another  clause  similar  to  the  Changes 
clause.  Under  its  authority,  the  Government; 

. . . may  (i)  decrease  the  property  provided  or 
to  be  provided  by  the  Govern.ment  under  this  contract, 
or  (ii)  substitute  other  Government  property  for 
property  to  be  provided  by  the  Governm.ent,  or  to  be 
acquired  by  the  contractor  for  the  Government,  under 
this  contract  [17:7:59]. 

Other  clauses  invoking  the  Changes  clause  pro- 
cedures include  the  Stop  Work  Order,  Suspension  of  Work, 
Differing  Site  Conditions,  and  the  Termination  for  Default- 
Damages  for  Delay-Time  Extension  clauses.  The  Stop  Work 


Order  clause  used  in  negotiated  fixed-price  contracts  allows 
the  Government  to  suspend  all  or  part  of  the  contract  per- 
formance for  up  to  ninety  days  (17:7:136).  Under  the 
authority  of  the  Suspension  of  Work  clause  used  in  construc- 
tion contracts, 

. . . the  Contracting  Officer  may  order  the  con- 
tractor in  writing  to  suspend,  delay,  or  interrupt 
all  or  any  part  of  the  work  for  such  period  of  time 
as  he  may  determine  to  be  appropriate  for  the  con- 
venience of  the  Government  [17:7:295]. 

The  construction  contract  Differing  Site  Conditions  clause 
allows  the  Government  to  make  an  equitable  adjustment  in 
the  contract  price  brought  about  by  an  unusual  and  unsus- 
pected condition  of  the  site  (17:7:272).  Another  clause 
used  in  construction  contracts  is  the  Termination  for 
Default-Damages  for  Delay-Time  Extensions  clause  which 
gives  the  Government  the  authority  to  terminate  the  con- 
tractor's right  to  proceed  with  contract  performance  when 
he  does  not  diligently  pursue  his  work  (17:7:273). 

Bilateral  supplemental  agreement  for  performance 
outside  the  scope  of  the  contract.  Authority  to  modify  a 
contract  does  not  have  to  be  contained  within  the  contract. 

Sometimes  a contract  must  be  modified  for  reasons 
not  expressly  covered  by  its  provisions.  In  such  a 
case,  the  change  cannot  be  effected  without  the  con- 
sent of  the  contractor  and  must  be  accomplished  by 
agreement  of  both  parties  [l:XI-4]. 

Contract  modifications  providing  for  work  not  within  the 

scope  of  the  contract  may  significantly  increase  or 
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decrease  the  obligation  of  both  parties  and  may  require  new 
procurement  authority  (2:135).  Care  must  be  taken  by  con- 
tracting officers  to  insure  supplemental  agreements  for  per- 
formance outside  the  scope  of  the  contract  do  not  exceed 
the  intent  of  the  original  agreement.  Such  actions  could 
be  considered  new  procurements  and  require  new  competitive 
contracts  (l:XI-5-6). 


Constructive  change  order  based  on  some  informal 
action  on  the  part  of  the  Government  that  implies  a con- 
tract is  to  be  modified.  The  concept  of  constructive 
change  is  well  established  in  Government  contract  law 
(l:XI-26) . 

A constructive  change  order  has  been  defined  as  an 
oral  or  written  act  or  omission  by  the  Contracting 
Officer  or  other  authorized  Government  official  which 
is  of  such  a nature  that  it  is  construed  or  inferred 
to  have  the  same  effect  as  a formal  written  change 
order  under  the  Changes  clause  [2:146]. 

Reimer  indicates  that  a constructive  change  order  can  occur 

when  a contractor  performs  requested  work  outside  the  scope 

of  the  contract  or  has  received  an  incorrect  contract 

interpretation  from  the  contracting  officer  (12:624). 

Other  examples  of  constructive  change  orders  include: 

. . . furnishing  defective  Government  specifica- 

tions; requiring  adherence  to  delivery  schedules  when 
a contractor  is  entitled  to  a time  extension,  denying 
a contractor  the  opportunity  to  employ  a permissible 
method  or  sequence  of  work;  or  erroneously  requiring 
a contractor  to  perform  contrary  to  his  correct  inter- 
pretation of  the  contract  requirements  [17:26:20]. 
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In  addition,  the  courts  will  likely  rule  that  silence  on 
the  part  of  the  Government  will  constitute  a constructive 
change  order  if  the  contracting  officer  knowingly  allows 
contractor  action  to  continue  when  it  should  be  stopped 
(l:XI-27).  ASPR  specifically  states  that  Government  repre- 
sentatives should  avoid  all  conduct  that  would  indicate  an 
unauthorized  modification  of  the  contract  (17:26:20). 

Justification 

Contract  modifications  are  a major  part  of  the 
administrative  burden  of  managing  contracts.  Some  major 
defense  programs  have  had  more  than  2,000  contract  modifica- 
tions a year  (6:539).  For  example,  as  of  31  January  1970, 
the  F-111  program  had  accumulated  a total  of  3849  approved 
contract  changes  (9:32). 

Not  only  are  contract  modifications  a major  portion 
of  the  workload  of  contract  administration,  but  Fox's 
analysis  of  a 1970  RAND  study  indicates  that  contract  modi- 
fications increased  original  contract  costs  by  approximately 
40  percent  (6:379).  A review  of  current  literature  indi- 
cates some  effort  has  been  made  towards  explaining  and 
documenting  the  reasons  contracts  must  be  modified  (6;  11; 
12).  However,  an  analysis  of  the  instruments  used  to  bring 
about  these  modifications  appears  to  be  nonexistent.  The 
heavy  administrative  burden,  the  problem  of  cost  growth, 
and  the  extensive  amount  of  time  spent  on  contract 
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modifications  implies  a necessity  for  using  the  most  effi- 
cient contract  modification  instrument  available.  Contract 
modification  instruments  should  be  analyzed  in  terms  of 
flexibility,  manageability,  cost  effectiveness,  and  time- 
liness; and  an  attempt  should  be  made  to  identify  the  most 
efficient  contract  modification  instrument.  The  most  effi- 
cient instriiment  could  then  aid  in  reducing  the  administra- 
tive burden,  in  controlling  cost  growth,  and  in  providing  the 
best  weapon  system,  construction  project,  research  and 
development  effort,  service,  or  supply  item  available  for 
each  DOD  dollar  spent. 

Limitations 

This  research  effort  was  limited  to  those  contract 
modifications  that  are  not  normally  anticipated  when  the 
contract  is  consummated.  Examples  of  these  contract 
modifications  not  considered  are:  exercise  of  contract 
options  (17:7:68),  variations  in  quantity  (17:7:3),  "order 
for  supplies  or  services  not  otherwise  changing  the  terms 
of  contract  or  agreements  (e.g.,  delivery  orders  under 
indefinite  delivery  type  contracts)  [17:26:1],"  and  adminis- 
trative changes  (17:26:2).  Contract  modifications  brought 
about  by  termination  actions,  either  for  convenience  or 
default  (1:XIX-1  through  XIX-51)  were  also  not  examined. 

This  research  effort  was  concerned  with  contract 
modifications  that  are  considered  to  be  within  the  scope 
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of  the  contract  (6:363),  modifications  for  performance  out- 
side the  scope  of  the  contract  (l:XI-4),  and  the  informal 
actions  of  the  Government  (constructive  change  orders) 
that  are  construed  as  contract  modifications  by  the  con- 
tractor (9:94) . 

Anticipated  modifications  are  usually  planned  and 
provided  for  at  the  inception  of  the  contract.  The  con- 
tract modifications  that  are  unforeseen  are  the  ones  that 
can  lead  to  cost  growth,  increased  workloads,  and  ineffi- 
cient performance  (11:516-526). 

Objective  of  the  Study 

The  primary  objective  of  this  study  was  to  analyze 
instruments  currently  employed  by  DOD  contract  placement 
and  management  agencies  to  implement  contract  modifications. 
The  purposes  of  the  analysis  were  to  determine  which  instru- 
ments are  preferred  by  the  different  agencies;  and  to 
identify  the  most  flexible,  manageable,  cost  effective,  and 
timely  instruments  for  modifying  contracts. 

Research  Questions 

1.  What  available  contract  modification  instru- 
ments are  preferred  by  the  different  DOD  contract  placement 
and  management  agencies? 

2.  What  is  the  overall  efficiency  of  these  instru- 
ments in  terms  of  flexibility,  manageability,  cost  effec- 
tiveness and  timeliness? 
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CHAPTER  II 


METHODOLOGY 

Questionnaire 

VThile  the  obvious  starting  point  for  exploring 
problems  in  a particular  field  of  knowledge  is  a survey  of 
the  applicable  body  of  literature,  only  a small  portion  of 
the  existing  knowledge  is  put  into  writing.  Much  can  be 
gained  by  obtaining  current  information  from  people  knowl- 
edgeable in  the  field  of  study.  One  method  of  accomplish- 
ing this  is  to  conduct  a survey  and  solicit  ideas  and  facts 
from  the  experts  in  the  field  (5:84-85).  The  greatest 
strength  of  a survey  is  its  flexibility  (5:199).  "It  is 
the  only  practical  way  to  learn  many  types  of  information 
[5:199]."  A survey  may  be  conducted  on  a personal  one-to- 
one  basis  as  in  an  interview,  or  it  may  be  conducted  on 
an  impersonal  basis  by  the  use  of  a questionnaire  (5:199). 

A questionnaire  is  often  thought  to  mean  all  types 
of  survey  instruments  (5:199),  but  it  is  defined  in  the 
strict  technical  sense  as,  "a  written  or  printed  form  com- 
prising a series  of  questions  submitted  to  a number  of 
persons  to  obtain  data  for  a survey  or  report  [7:1104]." 

A questionnaire  was  chosen  to  conduct  the  proposed  research 
because  (1)  questionnaires  should  be  used  when  the  data 
needed  are  held  by  individuals,  (2)  the  data  are  attitudes 
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or  knowledge  of  these  individuals,  (3)  the  data  are  not 
complex  enough  to  require  a personal  interview,  (4)  the 
data  do  not  lend  themselves  to  ongoing  observations,  and 
(5)  the  data  can  be  adequately  obtained  by  use  of  the  ques- 
tionnaire (14:10).  A questionnaire,  like  any  measurement 
device,  has  advantages  and  disadvantages. 

Advantages  of  a Questionnaire 

The  impersonal  nature  of  a questionnaire — its 
standardized  wording,  its  standardized  instructions 
for  recording  responses — ensures  some  uniformity  from 
one  measurement  situation  to  another  [13:239]. 

Some  other  advantages  of  questionnaires  are  that 
they  are  less  expensive  than  other  forms  of  gathering  data 
(13:239;  5:282),  they  can  be  administered  to  a large  num- 
ber of  individuals  at  the  same  time,  they  can  cover  a wider 
area  and  obtain  more  information  (13:239),  they  allow  the 
respondent  to  take  more  time  to  consider  his  answers,  and 
they  are  more  impersonal  (5:283).  The  advantage  of  anonymity 
may  be  the  most  important  advantage  because  "...  respon- 
dents may  have  greater  confidence  in  their  anonymity,  and 
thus  feel  free  to  express  views  they  fear  might  be  dis- 
approved of  or  might  get  them  into  trouble  [13:240]." 

Disadvantages  of  a Questionnaire 

A questionnaire  is  not  without  disadvantages.  It 
cannot  probe  as  deeply  into  the  subject  matter  as  an 
extended  series  of  observations,  or  even  as  deeply  as  a 


personal  interview  (5:203).  The  major  disadvantage  of  a 
questionnaire  is  participant  nonresponse  (13:241-242; 
5:203).  However,  nonresponse  is  normally  associated  with 
mailed  questionnaires. 

To  help  overcome  the  disadvantage  of  nonresponse, 
the  questionnaires  were  administered  to  individuals  attend- 
ing the  Air  Force  Institute  of  Technology's  (AFIT)  School 
of  Systems  and  Logistics  continuing  education  procurement 
courses.  The  questionnaires  were  personally  given  to,  and 
collected  from,  the  respondents  by  the  researchers.  The 
researchers  estimated  that  this  procedure  would  provide  a 
response  rate  of  over  80  percent  as  compared  to  the  usual 
poor  response  rate  of  10  to  50  percent  for  mailed  question- 
naires (13: 241) . 

The  other  factors  that  can  adversely  influence  the 
return  of  a questionnaire  were  also  considered.  These 
factors  include  sponsorship,  attractiveness  of  the  format, 
length,  ease  of  completion,  and  the  nature  of  the  people  to 
whom  it  was  given  (13:241-242). 

The  survey  used  in  this  research  was  sponsored  by 
AFIT's  School  of  Systems  and  Logistics  with  approval  by 
Headquarters  USAF.  Changes  were  made  in  the  questionnaire 
after  it  was  tested  that  improved  the  format,  greatly 
decreased  its  length,  and  aided  in  its  ease  of  completion. 
The  professionalism  and  experience  of  the  DOD  procurement 


19 


personnel  who  completed  the  questionnaires  contributed  sig- 
nificantly to  the  response  rate  of  the  survey. 

Structure  of  the  Questionnaire 

Questionnaires  contain  questions  pertaining  to 
information  collected  for  the  research  effort  to  seek 
facts,  attitudes,  or  preferences.  These  types  of  questions 
are  known  as  "sought  data"  (5:208)  or  "content  items" 
(14:244).  The  other  main  type  of  question  in  a question- 
naire is  known  as  a "respondent  characteristic"  (5:208) 
or  "classification  item"  (14:48).  Classification  items 
allow  classification  of  respondents  and  analysis  of  informa- 
tion. 

A major  consideration  in  designing  content  item 
questions  is  to  insure  that  the  respondent  knows  what  ques- 
tion is  being  asked.  A vocabulary  that  is  common  to  the 
researchers  and  respondents  is  an  important  consideration 
(5:214).  Even  though  a group  may  have  knowledge  of  the 
material  being  covered,  there  may  still  be  special  uses  of 
the  terminology.  "This  will  require  either  defining  the 
term  as  you  intend  to  use  it,  or  avoiding  the  ambiguous 
terminology  [14:54]."  A concentrated  effort  was  made  to 
explain  any  potentially  ambiguous  term.  Question  clarity 
received  major  emphasis  (5:214). 
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Types  of  Questions 


"In  the  standard  interview  or  questionnaire,  ques- 
tions are  presented  with  exactly  the  same  wording,  and  in 
the  same  order,  to  all  respondents  [13:255]."  The  response 
structure  of  standardized  questions  can  either  be  open, 
with  free  choice  of  expression,  or  closed,  with  specified 
choices  (5:219).  Closed  response  questions  were  chosen 
because  they  are  standardizable,  are  easy  to  give,  are 
inexpensive  to  analyze,  and  because  the  nature  of  the 
responses  may  help  clarify  the  meaning  of  the  question  and 
permit  the  respondent  to  answer  quickly  (14:257-262). 

Closed  response  questions  are  better  when  there  is 
a clear  frame  of  reference,  the  respondents'  level  of 
information  is  predictable,  and  the  researcher  believes 
the  respondent  understands  the  topic  [5:219]. 

However,  closed  response  questions  are  not  without  their 
disadvantages.  They  may  force  a statement  when  the  respon- 
dent does  not  really  know,  they  do  not  allow  for  any  quali- 
fication of  the  answer,  and  there  may  be  a disparity  in  the 
frame  of  reference  for  all  respondents  (13:257-262).  The 
researchers  felt  the  disadvantages  of  closed  response  ques- 
tions could  be  overcome  by  the  specialized  nature  of  the 
respondent's  knowledge. 

Multiple  choice  questions  are  the  simplest  and  most 
common  question  format  (14:60-61).  Paired  comparison  ques- 
tions are  a specialized  form  of  multiple  choice  questions. 
"With  this  technique  the  respondent  can  express  his  attitudes 
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in  an  unambiguous  manner  by  making  a choice  between  two 
objects  [5:241]."  Paired  comparison  questions  make  a 
ranking  process  easy,  but  can  be  quite  lengthy  when  several 
items  have  to  be  ranked  (14:88). 

With  paired  comparing  there  is  a risk  that  the 
respondents  will  tire  to  the  point  that  they  will  give 
ill-considered  answers  or  even  refuse  to  continue. 
Opinions  differ  as  to  what  the  upper  limit  is,  but 
five  or  six  stimuli  are  not  unreasonable  when  the 
respondent  has  other  questions  to  answer.  If  the 
interview  consists  only  of  comparisons,  as  many  as  15 
stimuli  may  be  compared  [5:241]. 

The  pretest  questionnaire  (see  Appendix  B) , with  only  six 
stimuli,  was  developed  using  paired  comparison  questions. 

The  literature  indicated  that  paired  comparison  questions 
would  give  a more  differentiated  ranking  of  the  contract 
modification  instruments.  Response  rate  for  the  survey 
test  was  acceptable.  However,  detailed  interviews  with 
test  respondents  indicated  that  a simple  rank  order  response 
construction  would  render  the  desired  differentiated  rank- 
ing and  increase  the  response  rate. 

Like  the  comparative  scaling  approach  of  paired 
comparison,  the  rank  order  method  of  scaling  avoids  the 
ambiguity  of  having  a respondent  select  the  one  "best" 
response  (5:241).  The  rank  order  method  . .is  faster 
than  paired  comparison  and  is  usually  easier  and  more  moti- 
vating to  the  respondent  [5:244]."  The  number  of  stimuli 
that  can  be  easily  handled  by  the  rank  order  method  ranges 
from  five  to  ten  (5:244).  The  rank  order  questions  in  the 
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questionnaire  were  designed  for  the  respondent  to  assign 
the  number  one  to  the  most  preferred  stimuli  and  so  on  to 
the  number  six  to  the  least  preferred  stimuli  (see  Appendix 
A)  . 

Testing  of  Questionnaire 

Testing  of  the  questionnaire  before  it  is  adminis- 
tered to  the  selected  sample  is  a major  aspect  of  the  survey 
process  (5;  13;  14;  16).  A test  insures  that 

. . . the  questions  gather  information  that  meet 
the  objectives  of  the  survey;  all  important  phases 
of  the  survey  have  been  adequately  covered;  the  ques- 
tionnaire stimulates  respondent  cooperation;  the 
questions  are  in  satisfactory  order;  and  the  questions 
are  completely  understood  by  the  respondents  [16:2-3]. 

The  test  should  be  administered  under  similar  conditions 
as  the  actual  survey  will  be  administered  and  the  sample 
size  should  be  between  15  to  30  people  (14:39).  If  a major 
revision  is  made  after  a test/  then  another  test  should  be 
conducted  before  the  questionnaire  is  administered  (15: 

551) . The  test  research  questionnaire  was  administered  to 
33  procurement  personnel  attending  AFIT's  Contract  Law 
Course  on  30  November  1976.  Twenty-one  respondents  satis- 
factorily completed  the  questionnaire  for  a 64  percent 
response  rate.  Based  on  the  test  survey  results,  and 
interviews  with  the  respondents,  the  questionnaire  was 
changed  from  a paired  comparison  method  ranking  scale  to 
a rank  order  method  ranking  scale.  The  questionnaire  was 
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significantly  shortened  without  making  major  changes  in 
the  ranking  method  or  changing  the  question  content. 


Population  and  Sample 
Description  of  Population 

The  purpose  of  this  research  was  to  analyze  con- 
tract modification  instruments  used  within  the  DOD.  The 
population  under  study  consisted  of  all  DOD  procurement 
personnel  and  was  estimated  to  be  approximately  50,000 
(3:39).  The  scope  of  the  study  was  limited  to  military 
officers,  noncommissioned  officers,  and  equivalent  grade 
civilians  assigned  to  DOD  contract  placement  and  management 
agencies.  These  personnel  included  procuring  and  adminis- 
trative contracting  officers,  contract  administrators,  cost/ 
price  analysts,  contract  negotiators,  buyers,  procurement 
staff /management  personnel,  and  other  procurement  related 
DOD  personnel. 

Sample  Description 

The  population  sam.ple  for  the  research  consisted 
of  approximately  300  experienced  DOD  contract  placement, 
contract  management,  and  related  procurement  personnel 
attending  procurement  management  courses  at  the  Continuing 
Education  Division,  School  of  Systems  and  Logistics,  Air 
Force  Institute  of  Technology,  Wright-Patterson  AFB,  Ohio. 
Personnel  attending  the  AFIT  continuing  education  courses 
are  selected  from  all  DOD  procurement  agencies.  The 
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assumption  was  made  that  they  were  representative  of  their 
agencies,  and  that  their  views  and  opinions  may  be  general- 
ized to  their  respective  agencies  (the  population  under 
study) . 

This  source  of  respondents  was  selected  for  several 
reasons;  including  location  convenience,  ease  and  speed  of 
data  collection,  and  the  probability  of  high  questionnaire 
response  rate.  It  was  also  unnecessary  to  develop  mailing 
lists  or  mail  questionnaires. 

Type  of  Sampling 

Nonprobability  judgmental  sampling  was  employed  in 
this  research  (5:164).  According  to  Sheard  and  Gnauck  the 
pri.me  objective  in  sampling  is  to  select  a group  of  people 
who  will  provide  responses  which  can  be  considered  to  be 
representative  of  the  population  (14:157).  The  major 
determinants  of  representativeness  of  the  sample  are  the 
researcher's  knowledge  of  the  parameters  and  his  judgment 
in  identifying  people  with  those  characteristics  as  respon- 
dents for  the  study  (14:180). 

The  sample  size  and  composition  of  the  respondents 
determines  the  degree  to  which  the  research  findings  may 
be  generalized  to  the  population  (5:164).  It  was  antici- 
pated at  a minimum,  that  the  results  of  the  data  analysis 
may  be  generalized  to  the  DCD  procurement  agencies  repre- 
sented by  the  respondents,  depending  upon  the  number 
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representing  each  agency.  The  researchers'  optimistic 
objective  was  to  be  able  to  generalize  from  the  sample  to 
the  entire  population. 


The  research  questionnaire  contained  enough  demo- 
graphic information  to  allow  stratification  of  the  data  by 
DOD  component  and  agency,  experience,  and  job  as  necessary 
for  analysis. 

Data  Collection  Plan 

The  research  questionnaire  was  administered  during 
February  and  March,  1977.  Figure  1 is  a schedule  listing 
the  procurement  course,  the  date  the  questionnaire  was 
administered,  the  number  of  personnel  attending  the  course 
and  the  course  qualification  prerequisites  (18:4-31  to 
4-34).  The  course  qualification  prerequisites  are  included 
in  Figure  1 to  provide  data  source  validity. 

Each  respondent  in  the  sample  was  given  a package 
containing  the  questionnaire  and  a letter  of  instruction. 
Respondents  were  asked  to  complete  the  questionnaire  and 
return  it  to  their  course  instructor  at  the  following  class 
session.  In  order  to  further  assure  data  validity,  respon- 
dents were  asked  not  to  discuss  with,  nor  seek  assistance 
from,  their  classmates.  The  assumption  was  made  that  the 
respondents  answered  the  questionnaire  honestly  and  without 
assistance.  The  questionnaires  were  administered  near  the 
beginning  of  the  course  to  obtain  the  most  up-to-date 
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Date  Questionnaire  Number  of  Personnel 

Administered  Attending Course  Qualification  Prerequisites 
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Figure  1.  Questionnaire  Administration  Schedule 


Date  Questionnaire  Number  of  Personnel 

Procurement  Course  Administered  Attending  Course  Qualification  Prerequisites 
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information  and  to  avoid  any  influence  the  course  may  have 
on  the  responses  obtained. 

Data  Analysis 

Data  analysis  was  performed  on  the  questions  con- 
tained in  the  questionnaire  on  both  the  content  and  the 
classification  items.  Questionnaire  content  items  provided 
information  pertaining  to  facts,  attitudes,  and  preferences. 
Classification  items  were  primarily  used  to  classify 
respondents  for  the  purpose  of  analyzing  the  information 
provided. 

Content  Items 

1.  Flexibility — the  degree  to  which  a contract 
modification  instrument  is  useful  in  responding  to  new 
situations . 

2.  Manageability — the  degree  to  which  a contract 
modification  instriiment  is  easily  implemented  and  con- 
trolled. 

3.  Cost  Effectiveness — the  degree  to  which  a con- 
tract modification  instrument  is  least  costly  to  administer 
and  has  the  least  potential  for  contract  cost  growth. 

4.  Timeliness — that  contract  modification  instru- 
ment requiring  the  least  time  to  initiate  and  complete. 

Classification  Items 

1 . Respondent's  Duty  Organization. 

2 . Respondent's  Grade  Level. 
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3. 

4. 

Respondent's  Total  Federal  Service. 

Respondent's  Total  Procurement  Experience. 

5. 

Respondent's  Current  Job  Description. 

6. 

Price  Range  of  Contract  Modifications. 

7. 

Type  of  Contract  Modifications  Accomplished. 

8. 

Amount  of  Time  Spent  on  Contract  Modifications. 

Questionnaire  Content 

The  research  questionnaire  contained  three  parts. 
Part  one  contained  demographic  information  designed  to  per- 
mit selective  stratification  of  the  data  by  respondent's 
background,  grade,  and  organization.  Part  two  of  the 
research  questionnaire  was  designed  to  answer  the  first 
research  question.  Part  three  of  the  research  question- 
naire was  designed  to  answer  the  second  research  question. 

Research  Question  One 

Question  number  10  of  the  questionnaire  (see 
Appendix  A)  was  designed  to  answer  research  question  one. 

A frequency  distribution  was  tabulated  indicating  contract 
modification  instrument  preference  by  DOD  contract  place- 
ment and  management  agency.  The  modification  instrument 
with  the  highest  selection  rate  for  each  agency  was  con- 
sidered the  most  preferred  by  that  agency.  In  addition, 
joint  frequency  distributions  were  accomplished  on  ques- 
tions 6 and  7 with  questions  9,  10,  and  11  in  an  attempt 
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to  stipulate  the  conditions  for  using  a particular  con- 
tract modification  instrument. 

To  validate  the  survey  findings,  75  actual  contract 
modifications  were  examined  at  the  2750th  Air  Base  Wing, 
Procurement  Division,  Wright- Patterson  AF3,  Ohio,  the 
Defense  Electronic  Supply  Center  (DESC) , Dayton,  Ohio,  and 
the  Defense  Contract  Administration  Management  Area 
(DCASMA) , Dayton,  Ohio.  The  researchers  reviewed  the  con- 
tract modifications  and  supporting  documents  to  determine 
the  type  of  contract  modification  instr\ments  being 
employed.  The  contract  modification  instruments  actually 
used  by  the  agencies  visited  were  compared  to  the  question- 
naire responses  to  question  10  from  the  saime  agency  classi- 
fication. 

Research  Question  Two 

Respondents  were  asked  to  rank  order  the  six  con- 
tract modification  instruments  for  each  of  four  variables. 
The  rank  order  technique  permitted  the  respondents  to 
express  their  opinions  in  an  unambiguous  manner  (5:24). 

The  method  of  rank  ordering  produced  an  ordinal  level  rank 
ordering  of  the  six  contract  modification  instruments  in 
terms  of  the  most  flexible,  manageable,  cost  effective,  and 
timely  instrximent  for  modifying  contracts  (5:244).  The 
question  stem  for  each  variable  instructed  the  respondent 
to  place  the  number  one  beside  the  contract  modification 
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instrument  he  felt  to  be  most  representative  of  the  vari- 
able identified.  The  respondent  was  instructed  to  place 
the  number  two  beside  the  instrument  he  considered  the  next 
most  representative  and  so  on  to  number  six,  for  the  instru 
ment  he  considered  the  least  representative  of  the  vari- 
able. 

Sample  results  for  each  instrument  were  then 
totaled.  Because  the  number  one  was  assigned  to  the  instru 
ment  most  representative  of  the  variable,  the  contract 
modification  instrument  with  the  lowest  overall  total  was 
ranked  as  number  one.  The  instrument  with  the  greatest 
overall  total  was  ranked  as  number  six.  The  tabulated 
rankings  of  the  six  contract  modification  instruments  were 
then  combined  and  analyzed  for  statistical  significance 
using  the  Friedman  Two-Way  Analysis  of  Variance  by  Ranks 
Test  and  the  Kendall  Coefficient  of  Concordance  Test. 

Test  Design  I 

The  Friedman  Two-Way  Analysis  of  Variance  by  Ranks 
Test,  tests  the  hypothesis  that  various  conditions,  or  in 
this  research  project  the  various  contract  modification 
instruments  ranking,  came  from  the  same  population  (15:166) 
If  the  variables  are  independent  of  the  contract  modifica- 
tion instruments,  the  rank  totals  in  each  column  will  be 
approximately  equal,  but  if  the  variables  are  dependent  on 


the  instruments,  the  rank  totals  will  vary  among  the  columns 
(15:16).  The  specific  hypothesis  test  conducted  was: 

Hq:  The  contract  modification  instruments  are  inde- 
pendent of  the  variables; 

H^:  The  contract  modification  instriaments  are 
dependent  on  the  variables. 

The  Friedman  Test  can  be  conducted  using  the  same 
ranked  ordinal  data  obtained  from  the  survey,  with  the 
variables  of  flexibility,  manageability,  cost  effectiveness, 
and  timeliness  considered  the  row  effects;  and  the  various 
contract  modification  instruments  considered  the  column 
effects.  The  same  ranking  order  will  be  maintained  of  from 
highest  to  lowest,  with  the  highest  ranked  instrument 
receiving  a one  and  the  lowest  ranked  instrument  receiving 
a six.  In  performing  the  Friedman  Test,  "it  is  immaterial 
whether  the  ranking  is  from  lowest  to  highest  score  or  from 
highest  to  lowest  score  [15:167]." 

The  Friedman  Two-Way  Analysis  of  Variance  by  Ranks 
2 

Test,  Xr  , is  computed  from  a two-way  table  having  N rows 
and  k columns.  Each  row  will  indicate  an  independent  rank- 
inc  of  from  one  to  k.  "The  Friedman  test  determines  whether 
the  rank  totals  (Rj)  differ  significantly  [15:165]."  Xr^ 
is  computed  from  the  following  formula: 
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2 

where  Xr  = Friedman  Two-Way  Analysis  of  Variance  by 
Ranks ; 

N = number  of  rows  (variables) ; 
k = number  of  columns  (instruments);  and 
Rj  = sum  of  each  individual  column  (15:168). 

2 

Test  of  Statistical  Significance  of  Xr  . When  the 

values  of  N and  k are  equal  to  or  greater  than  N=4  and  k=3, 

2 

the  sampling  distribution  of  Xr  approximates  a chi-square 

distribution  with  degrees  of  freedom  (d.f.)  = k-1  (15:168). 

2 

Xr  can  be  compared  to  the  appropriate  value  at  the  desig- 
nated alpha  level  obtained  from  Table  K of  Clarke  and 

2 

Schkade  (4:845).  If  the  computed  Xr  is  equal  to  or  greater 
than  the  particular  level  of  significance,  then  may  be 
rejected  (15:168).  The  rejection  of  the  null  hypothesis 
helps  establish  the  statistical  dependency  between  the 
contract  modification  instrument  and  the  variables  by  which 
it  is  being  ranked.  In  addition  to  the  Friedman  Test,  the 
data  were  analyzed  by  the  Kendall  Coefficient  of  Concordance 
Test. 

Test  Design  II 

The  Kendall  Coefficient  of  Concordance  Test  was  also 
performed  to  determine  the  degree  of  association  among  the 
variables  (15:229).  Significant  statistical  association 
would  support  the  research  question  objective  of  identifying 
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the  particular  contract  modification  instrument  that  is 
most  efficient  based  on  flexibility,  manageability,  cost 
effectiveness  and  timeliness.  Specifically,  the  hypothesis 
test  conducted  was: 

Hg!  There  is  independence  among  the  variables; 

There  is  no  independence  among  the  variables. 

To  compute  the  Kendall  Coefficient  of  Concordance: 
W,  the  sura  of  ranks,  Rj , in  each  column  of  a k by  M table 
must  first  be  found.  The  means  value  of  is  then  found 
by  dividing  the  sum  of  all  R ^ ' s by  N.  The  individual  R ^ ' s 
can  then  be  expressed  as  deviations  from  the  mean  value. 

The  sum  of  squares  of  the  deviations,  s,  of  the  R^ ' s can 
then  be  computed.  W can  then  be  computed  from  these  vari- 
ables : 

s 

W - 

l/12k^ 


Definition  of  Variables: 

W = Kendall  Coefficient  of  Concordance ; 

R.  = the  sum  of  each  individual  column; 

3 

k = number  of  rows  (variables) ; 

N = objects  that  are  being  ranked  (instr'aments ) ; 
s = sum  of  squares  of  the  deviations  from  the 
mean  R^ , where, 
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l/12k^  (N^-N) 


= the  maximum  possible  sum  of  the 
squared  deviations,  value  of  s,  if 
there  is  perfect  agreement  among 
the  variables  (15:231). 


Test  of  Statistical  Significance  of  W.  "We  may  test 
the  significance  of  any  observed  value  of  W by  determining 
the  probability  associated  with  the  occurrence  under  Hq  of 
a value  as  large  as  the  s with  which  it  is  associated  [15: 
235]."  The  distribution  of  s under  has  been  provided  in 
Table  R,  page  286,  of  Seigel's  book  for  values  of  s for 
W's  significant  at  the  .05  and  .01  levels.  Table  R can 
be  used  for  k values  of  from  3 to  20,  and  N values  of  from 
3 to  7 (15:236).  "If  an  observed  s is  equal  to  or  greater 
than  that  shown  in  Table  R for  a particular  level  of  sig- 
nificance, then  Hq  may  be  rejected  at  that  level  of  signifi- 
cance [15: 236] . " 


Interpretation  of  W.  Kendall  indicates  that  the 
best  esti.mate  of  the  ranking  of  t.he  objects  is  given  by  the 
order  of  the  various  sums  of  the  ranks  when  W is  signifi- 
cant (15; 238).  Given  the  order  of  the  sum  of  ranks  when 
W is  significant,  the  instrument  with  the  lowest  overall 
sum  can  be  accepted  as  the  most  efficient. 


Suirjnarv  of  Assumotions  and  Limitations 


Assumptions 

1.  The  respondents  from  each  contract  placement 
and  management  agency  represented  a homogeneous  group  with 
respect  to  their  viewpoint  about  contract  modification 
instruments. 

2.  A questionnaire  was  an  appropriate  method  of 
generating  the  data  necessary  to  address  the  research  ques- 
tions . 

3.  A high  (over  80  percent)  questionnaire  response 
rate  was  expected. 

4.  Respondents  answered  the  questionnaire  honestly 
and  conscientiously. 

5.  Respondents  had  the  knowledge  and  ability  neces- 
sary to  answer  questions  with  a valid  opinion. 

Limitations 

1.  The  composition  of  respondent  agencies  may  limit 
the  degree  to  which  the  research  conclusions  may  be  general- 
ized. 

2.  The  time  allotted  for  distributing  and  gather- 
ing data  was  limited  to  February  and  March,  1977. 

3.  Sample  composition  may  limit  strength  of  popu- 
lation generalization. 
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CHAPTER  III 


DATA  PRESENTATION 
Introduction 

Chapter  III  contains  an  analysis  of  the  results  of 
the  survey.  It  includes  a review  of  the  questionnaire 
response  rate,  the  constructed  data  files,  and  the  computer 
programs  used  for  analysis.  In  addition,  it  includes 
detailed  analyses  of  the  following  items. 

1.  the  one-way  frequency  distributions  on  ques- 
tionnaire questions  1 through  11; 

2.  the  joint  frequency  distributions  on  question- 
naire questions  6 and  7 with  questions  9,  10,  and  11  used 
to  answer  Research  Question  One; 

3.  the  Friedman  Two-Way  Analysis  of  Variance  by 
Rank  and  the  Kendall  Coefficient  of  Concordance  statistical 
tests  used  to  answer  Research  Question  Two;  and 

4.  the  one-way  frequency  distributions  and  joint 
frequency  distributions  on  t.he  actual  observed  contract 
modification  data  used  to  support  the  survey  data  pertain- 
ing to  Research  Question  One. 

Questionnaire  Response  Rate 

A total  of  299  questionnaires  were  distributed 
during  February  and  March,  1977  (see  Figure  1) . Of  the  299 
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distributed  questionnaires,  253  were  returned  for  an  overall 
response  rate  of  84.62  percent.  The  actual  response  rate 
exceeded  the  estimated  response  rate  by  4.62  percent.  As 
in  the  case  of  all  surveys,  not  all  of  the  returned  ques- 
tionnaires were  usable. 

Question  number  4 of  the  questionnaire  contained  a 
response  for  "no  procurement  related  experience."  A care- 
ful screening  of  the  253  returned  questionnaires  indicated 
that  59  were  from  individuals  without  procurement  experi- 
ence. The  other  194  questionnaires  were  considered  usable 
for  the  purpose  of  analysis.  These  194  questionnaires  pro- 
vided a usable  response  rate  from  returned  questionnaires 
of  76.68  percent,  and  a usable  response  rate  from  the  total 
number  distributed  of  64.88  percent.  Data  files  for 
analysis  were  constructed  from  the  identified  usable  ques- 
tionnaires . 

Data  Files 

Data  files  were  manually  compiled  and  entered  on 
the  Computational  Resources  for  Engineering  and  Simulation 
Training  and  Education  (CREATE)  computer  system  (see 
Appendix  D) . Six  files  were  constructed  from  the  survey 
data,  and  one  from  the  actual  observed  contract  modifica- 
tion data. 

Survey  Data  Files.  The  construction  of  the  survey 
data  files  was  based  on  the  responses  selected  on  the 
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usable  questionnaires.  The  questionnaires  were  consecu- 
tively numbered  beginning  with  number  101  through  number 
294.  The  questionnaire  nxunbers  were  then  used  as  the  line 
numbers  for  the  data  files.  Maintaining  the  same  numbering 
system  provided  easy  cross-reference  from  the  data  files 
to  the  actual  questionnaires  for  the  purpose  of  quality 
control. 


After  the  line  number  entry,  the  responses  to 
questions  1 through  11  were  entered  as  a block  of  alpha- 
numeric data.  The  responses  to  questions  12  through  15  all 
contained  six  numeric  entries.  The  six  numeric  responses 
for  each  of  questions  12  through  15  were  entered  as  a 
separate  block.  This  made  four  blocks  of  numbers  each  con- 
taining six  numeric  entries.  A single  numeric  "dummy" 
value  of  one  was  entered  as  the  last  variable  on  each  indi- 
vidual file.  The  "dummy"  value  was  used  to  count  the  num- 
ber of  files. 

In  addition  to  the  main  data  file  containing  all  of 
the  usable  questionnaires,  separate  data  files  v.ere  com- 
piled for  Air  Force,  Army,  Navy,  Defense  Logistics  Agency 
(DLA) , and  other  Federal  Agency  respondents.  The  breakout 
for  the  separate  agencies  was  accomplished  using  the 
responses  given  for  question  one.  For  example,  if  the 
response  to  question  one  was  "a,"  "b,"  or  "c,"  then  the 
questionnaire  was  entered  into  the  Air  Force  data  file. 

It  was  felt  that  further  breakout  as  to  agency  would  not  be 
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meaningful  due  to  the  number  of  respondents.  The  primary 
purpose  for  establishing  separate  data  files  was  to  provide 
information  to  answer  Research  Question  Two.  The  line 
number /questionnaire  number  integrity  was  again  maintained 
for  the  purpose  of  quality  control. 

As  in  the  main  data  file,  the  "dummy”  value  vari- 
able was  used  to  count  the  entries  in  each  separate  file. 
There  were  78  respondents  from  the  Air  Force,  54  from  the 
Army,  35  from  the  Navy,  21  from  the  DLA,  and  6 from  other 
federal  agencies. 

Actual  Data  File.  A data  file  was  created  from  the 
actual  observed  contract  modifications.  Unlike  the  survey 
data  files,  the  75  actual  observations  contained  only  four 
variable  entries.  The  four  variables  were  agency  observed, 
contract  modification  instrument  used,  price  range  of  the 
contract  modification,  and  type  of  contract  modification. 
The  data  file  entries  were  all  coded  as  numeric  entries, 
and  corresponded  to  the  responses  entered  in  the  survey 
data  file. 

Computer  Programs 

Four  computer  programs  were  used  to  analyze  the 
data  collected  (see  Appendix  C) . Three  of  the  programs 
were  written  using  the  Statistical  Package  for  the  Social 
Sciences  (SPSS)  language.  The  fourth  program  was  a simple 
Formula  Translation  (FORTRAN)  program.  The  SPSS  programs 
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were  used  to  obtain  one-way  frequency  distributions  and 
joint  frequency  distributions  on  the  survey  and  actual 
data.  The  FORTRAN  program  provided  a sum  of  the  rankings 
for  each  contract  modification  instrviroent  of  questions  12 
through  15.  The  rankings  were  used  to  perform  the  sta- 
tistical tests.  All  programs  were  entered  and  run  on  the 
CREATE  system  from  a remote  terminal. 

Program  1.  Program  1 computed  and  presented  one- 
way frequency  distribution  tables  using  SPSS  subprogram 
FREQUENCIES  (10:194).  The  variables  were  presented  on  the 
output  tables  along  with  the  absolute  frequency  of  their 
occurrence  and  the  relative  frequency,  in  percentages,  of 
their  occurrence.  In  addition  to  the  tables,  a histogram 
showing  a graphic  display  of  each  variable  was  presented. 

A single  frequency  distribution  was  requested  for 
each  question,  1 through  11.  Then  by  using  the  RECODE  and 
‘SELECT  IF  commands,  frequency  distributions  were  run  on 
questions  2 through  11  for  each  agency. 

Program  2.  Program  2 computed  and  presented  joint 
frequency  distribution  tables  using  SPSS  subprogram  CROSS- 
TABS (10:218).  CROSSTABS  can  be  used  to  compute  two-way 
to  n-way  joint  frequency  tables.  Program  2 produced  a 
three-way  cross-tabulation  on  questions  6 and  7 with  ques- 
tions 9,  10,  and  11.  Program  2 was  constructed  essentially 
the  same  as  Program  1,  with  the  task  command  CROSSTABS 
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replacing  the  command  FREQUENCIES.  Also,  the  RECODE  and 
*SELECT  IF  commands  provided  selected  products  by  agency. 

The  cross- tabulation  tables  produced  were  matrix 
tables  with  the  variables  of  question  6 presented  on  the 
vertical  axis  and  the  variables  of  question  7 presented  on 
the  horizontal  axis.  A table  was  produced  for  each  vari- 
able selected  of  the  controlling  question  (9,  10,  or  11), 
showing  the  number  and  associated  percentages  of  the 
response  selected. 

Program  3.  Program  3 was  constructed  to  analyze  the 
actual  observed  contract  modification  data.  The  smaller 
number  of  variables  and  observations  allowed  the  task  com- 
mands FREQUENCIES  and  CROSSTABS  to  be  combined  in  one  pro- 
gram. The  one-'way  frequency  distributions  and  joint  fre- 
quency distributions  outputs  were  produced  in  the  same  for- 
mat as  Programs  1 and  2,  and  corresponded  to  the  same  vari- 
ables . 


Program  4 . Program  4 was  written  in  FORTRAN.  It 
used  a DO  LOOP  to  sum  the  values  of  the  ranked  variables  of 
questions  12  through  15.  Each  data  file  was  computed  indi- 
vidually by  use  of  a temporary  call  command.  Program  4 also 
provided  the  total  number  of  cases  in  each  file  by  summing 
the  "dummy"  variable. 
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One-Way  Frequency  Distributions 


One-way  frequency  distributions  are  presented  for 
each  question  in  Tables  1 through  11.  Absolute  frequencies 
for  each  agency,  and  absolute  frequencies  for  the  overall 
responses  along  with  relative  frequencies  are  shown. 

Question  one  (see  Table  1)  of  the  survey  question- 
naire was  designed  to  permit  breakdown  of  the  respondent's 
duty  organization  into  subcomponents  such  as  major  command. 
However,  due  to  the  limited  number  of  respondents  in  some 
of  the  subcomponents,  the  responses  were  combined  into  five 
categories . 


TABLE  1 

DUTY  ORGANIZATION 


Absolute 

Relative 

Agency 

Frequency 

Frequency  (%) 

Department  of  the  Air  Force 

78 

40.2 

Department  of  the  Army 

54 

27.8 

Department  of  the  Navy 

35 

18.0 

Defense  Logistics  Agency 

21 

10.8 

Other  Federal  Agencies 

6 

3.1 

194 

100. 0 

Table  2 indicates  that  of  the  procurement  personnel 
responding  to  the  questionnaire,  79.9  percent  were  civilian 
employees  with  the  majority  in  the  grades  of  GS-8  through 
GS-12.  The  majority  of  militairy  respondents  were  Air  Force 
officers  in  grades  0-1  through  0-3. 
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The  distribution  of  Federal  service  time  was  fairly 
even  over  all  categories  (see  Table  3) . The  Army  appeared 
to  be  sending  a larger  number  of  personnel  to  school  with 
less  than  three  years  of  Federal  service  than  the  other 
agencies. 

A comparison  of  questions  3 and  4 (see  Tables  3 and 
4)  indicated  that  many  of  the  respondents  have  cross- 
trained  into  procurement  from  other  career  fields.  For 
instance,  only  20.6  percent  of  the  respondents  had  less 
than  three  years  of  Federal  service  but  40.7  percent  had 
less  than  three  years  of  procurement  experience.  Similarly, 
16.5  percent  of  the  respondents  had  over  20  years  of 
Federal  service  but  only  5.2  percent  had  over  20  years  of 
procurement  experience. 

Table  5 indicates  that  a majority,  59.3  percent, 
of  the  respondents  considered  themselves  either  Contract 
Negotiators,  Buyers,  or  Contract  Administrators.  This 
represented  procurement  personnel  most  likely  to  be  modify- 
ing contracts. 

Most  of  the  respondents  (87.1  percent)  indicated 
that  they  most  frequently  handled  contract  modifications 
in  the  no  cost  through  $100,000  price  range  (see  Table  6). 

It  was  interesting  to  note  that  three  respondents  indicated 
they  most  frequently  accomplished  contract  modifications  in 
the  over  $1,000,000  price  range. 
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194  100. 


Changes  in  contract  specifications  and/or  drawings 
was  chosen  as  the  predominant  reason  for  having  to  modify 
contracts  (see  Table  7) . The  distribution  of  responses  to 
this  question  provided  an  indication  of  where  management 
attention  may  be  required  if  the  number  of  contract  modifi- 
cations is  to  be  reduced. 

A majority,  65  percent,  of  the  respondents  indi- 
cated that  they  spend  20  percent  or  less  of  t.heir  working 
time  on  contract  modificaticn  (see  Table  8) . However, 

37.1  percent  of  the  Navy  personnel  indicated  that  they 
spe.nd  from  41  to  100  percent  of  their  time  on  contract 
modifications . 

A bilateral  supplemental  agreement  issued  under 
the  authority  of  the  Changes  clause  was  the  most  frequently 
used  contract  modificaticn  instrument  (see  Table  9) . This 
selection  was  followed  respectively  by  a bilateral  supple- 
ment agree.ment  issued  u.nder  a contract  provision  other  than 
the  Changes  clause;  a unilateral  change  order  issued  under 
the  Changes  clause;  a bilateral  supplemental  agreement 
issued  for  performance  outside  the  scope  of  the  contract; 
a unilateral  change  order  issued  under  a contract  provision 
other  than  the  Changes  clause;  do  not  know;  and  a construc- 
tive change  order.  The  one-way  frequency  distributions 
indicated  a general  agreement  as  to  the  order  of  modifica- 
tion instruments  used  most  frequently.  Bilateral  methods 


L 


51 


52 


53 


1 


TABLE  9--Continued 


Bilateral  supplemental  agreement  issued 
of  a contract  provision  other  than  the 

under  authority 
Changes  clause 

USAP 

Absolute 

Frequency 

18 

Relative 
Frequency  (%) 

23.1 

Army 

15 

27.8 

Navy 

10 

28.6 

DLA 

11 

52.4 

Other  Federal 

4 

66.7 

All  Agencies 

58 

29.  9 

Bilateral  supplemental 

agreement  for 

performance  outside 

the  scope 

of  the  contract 

USPF 

Absolute 

Freauency 

6 

Relative 
Frequency  (%) 

7.7 

Army 

4 

7.4 

Navy 

3 

8.6 

DLA 

- 

- 

Other  Federal 

- 

- 

All  Agencies 

13 

6.7 

Constructive  change  order  based  on  some  informal 
action  on  the  part  of  the  Government  that 
implies  a contract  is  to  be  modified 


USAF 

Absolute 

Frequency 

2 

Relative 
Frequency  (%) 

2.6 

Army 

- 

“ 

Navy 

- 

— 

DLA 

- 

- 

Other  Federal 

- 

- 

All  Agencies 

2 

1.0 

Do  not  know 


USAF 

Absolute 

Frequency 

5 

Relative 
Frecuency  (%) 

6.4 

Army 

3 

5.6 

Navy 

1 

2.9 

DLA 

- 

Other  Federal 

- 

- 

All  Agencies 

9 

4.6 
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were  used  more  frequently  by  all  agencies.  However,  there 
was  a higher  use  of  unilateral  methods  by  DLA  respondents. 

A relatively  large  percentage,  20.1  percent,  of  the 
respondents  indicated  that  they  did  not  know  which  contract 
modification  instrument,  if  any,  was  preferred  for  use  by 
their  agencies  {see  Table  10) . The  majority  of  all  respon- 
dents, except  DLA  respondents,  indicated  an  agency  prefer- 
ence for  bilateral  supplemental  agreements  issued  under  the 
Changes  clause.  DLA  respondents  indicated  an  agency  pre- 
ference for  bilateral  supplemental  agreements  issued  under 
contract  provisions  other  than  the  Changes  clause.  None  of 
the  respondents  indicated  that  their  agencies  preferred 
the  use  of  a constructive  change  order. 

A majority  of  the  respondents  (56.2  percent,  indi- 
cated a personal  preference  for  bilateral  supplement  agree- 
ments issued  either  under  the  Changes  clause  or  under  other 
contract  provisions  (see  Table  11) . However,  a relatively 
large  percentage,  24.2  percent,  of  respondents  indicated  no 
preference. 


Joint  Frequency  Distributions 
Joint  frequency  distributions  were  generated  by  use 
of  Program  2 on  all  survey  data  files.  However,  because  of 
the  low  number  of  respondents  from  other  Federal  Agencies, 
questionnaires  completed  by  these  respondents  were  not 
analyzed  as  a separate  file.  A total  of  86  cross- tabulation 
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TABLE  10 


CONTRACT  MODIFICATION  INSTRUMENT,  AGENCY  PREFERENCE 


Unilateral 

change 

order  issued  under  authority  of 

the 

Changes  clause 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

8 

10.3 

Army 

4 

7.4 

Navy 

1 

2.9 

DLA 

4 

19.0 

Other  Federal 

- 

- 

All  Agencies 

17 

8.  8 

Bilateral 

supplemental  agreement  issued 

under 

authority 

of  the  Changes  clause 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

33 

48.7 

Army 

17 

31.5 

Navy 

17 

48.6 

DLA 

3 

14.3 

Other  Federal 

3 

50.0 

All  Agencies 

78 

40.2 

Unilateral  change  order  issued  under  authority  of  a 

contract  provision 

other  than  the  Changes 

clause 

Absolute 

Relative 

Frequency 

Freque.ncy  (%) 

USAF 

1 

1.3 

Army 

1 

1.9 

Navy 

- 

DLA 

3 

14.3 

Other  Federal 

- 

- 

All  Agencies 

5 

2.6 
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TABLE  10 — Continued 


Bilateral  supplemental  agreement  issued 
of  a contract  provision  other  than  the 

under  authority 
Changes  clause 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

12 

15.4 

Army 

14 

25.9 

Navy 

9 

25.7 

DLA 

11 

52.4 

Other  Federal 

2 

33.3 

All  Agencies 

48 

24.7 

Bilateral  supplemental 

agreement  for 

performance  outside 

the  scope 

1 of  the  contract 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

3 

3.8 

Army 

1 

1.9 

Navy 

3 

8.6 

DLA 

- 

- 

Other  Federal 

- 

- 

All  Agencies 

7 

3.  6 

Do  not  know 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

16 

20.  5 

Army 

17 

31.5 

Navy 

5 

14.3 

DLA 

- 

- 

Other  Federal 

1 

16.7 

All  Agencies 

39 

20.  1 
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TABLE  11 


CONTRACT  MODIFICATION  INSTRUMENT  PREFERRED  BY  RESPONDENT 


Unilateral  change  i 
the 

crder  issued  under  authority  of 
Changes  clause 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

7 

9.0 

Army 

10 

18.5 

Navy 

2 

5.7 

DLA 

5 

23.8 

Other  Federal 

2 

33.3 

All  Agencies 

26 

13.4 

Bilateral  supplemental  agreement 

. issued 

under 

authority 

of  the  Changes 

clause 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

33 

42.3 

Army 

15 

2 

DLA 

5 

23.8 

Other  Federal 

1 

16.7 

All  Agencies 

70 

36.1 

Unilateral 

change  order  issued  under  authority  of  a 

contract 

provision 

other  than  the 

Changes 

clause 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

3 

3.8 

Army 

2 

3.7 

Navy 

1 

2.9 

DLA 

- 

- 

Other  Federal 

- 

- 

All  Agencies 

6 

3.1 

59 
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TABLE  ll--Continued 


Bilateral  supplemental  agreement  issued  under  authority 
of  a contract  provision  other  than  the  Changes  clause 


Absolute 

Relative 

Frequency 

Freauency  (%) 

USAF 

11 

14.1 

Army 

11 

20.4 

Navy 

8 

22.9 

DLA 

8 

38.1 

Other  Federal 

1 

16.7 

All  Agencies 

39 

20.1 

Bilateral  supplemental  agreement  for  performance  outside 

the 

scope  of  the  contract 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

1 

1.  3 

Army 

2 

3.7 

Navy 

1 

2.9 

DLA 

- 

Other  Federal 

- 

- 

All  Agencies 

4 

2.1 

Constructive 

change  order  based  on  some 

informal 

action  on 

the 

part  of  the  Government 

that 

implies 

a 

contract  is  to  be  modified 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

- 

- 

Army 

1 

1.  9 

Navy 

— 

DLA 

- 

- 

Other  Federal 

1 

16.7 

All  Agencies 

2 

1.0 

Have  no  preference 

Absolute 

Relative 

Frequency 

Frequency  (%) 

USAF 

23 

29.5 

Army 

13 

24.1 

Navy 

7 

20.0 

DLA 

3 

14.3 

Other  Federal 

1 

16.7 

All  Agencies 

47 

24.2 
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tables  were  produced  for  the  variables  of  questions  9,  10, 
and  11.  It  would  not  have  been  feasible  to  analyze  all  of 
the  tables  because  of  the  limited  number  of  responses  in 
some  categories.  Therefore,  only  the  cross-tabulation 
tables  for  the  contract  modification  instrument  selected 
most  frequently  for  each  question  9,  10,  and  11  are  pre- 
sented. There  are  a total  of  15  tables  shown — one  for 
each  question  9,  10,  and  11  for  All  Agencies,  Air  Force, 
Army,  Navy,  and  DLA. 

Question  9:  Contract  Modification 
Instr^oment  Most  Frequently  Used 

The  contract  modification  instrument  most  fre- 
quently used  by  All  Agencies,  Air  Force,  Army,  and  Navy 
was  found  to  be  the  bilateral  supplemental  agreement  issued 
under  authority  of  the  Changes  clause.  The  DLA  respondents 
indicated  that  the  bilateral  supplemental  agreement  issued 
under  authority  of  a contract  provision  other  than  the 
Changes  clause  was  the  most  frequently  used. 

All  Agencies.  Seventy-one  of  the  one  hundred 
ninety-four  total  respondents  selected  the  bilateral  supple- 
mental agreement  issued  under  authority  of  the  Changes 
clause  for  a 36.6  percent  selection  rate.  The  majority 
that  chose  this  instrument  indicated  that  28.2  percent  of 
the  time  they  made  changes  in  specifications  in  the  price 
range  of  $10,000  or  less  (See  Table  12). 
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TABLE  12 


CONTRACT  MODIFICATION  INSTRUMENT  USED  MOST  FREQUENTLY 

BY  ALL  AGENCIES 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

Pack/ 

Ship 

Method 

Qty 

GFM 

Row 

Total 

3 

1 

0 

0 

0 

4 

No 

Cost 

4.2 

1.4 

0. 

0. 

0. 

5.  6 

$10,000 

20 

9 

1 

3 

1 

0 

33 

or 

Less 

28.2 

12.7 

1.4 

4.2 

0. 

Over 

$10,000 

17 

4 

0 

3 

1 

25 

thru 

$100, 000 

23.9 

5.6 

0. 

4.2 

1.4 

35.2 

Over 
$100, 000 

5 

1 

0 ! 

1 

2 

0 

8 

thru 

$i,ooo,ooc 

7.0 

0. 

1.4 

2.8 

0. 

11. 3 

1 

0 

0 

i 

0 

0 

1 

Over 

$1,000,00C 

1.4 

1 

0. 

0. 

0. 

0. 

1.4 

Column 

46 

14 

2 

8 

1 

71 

Total 

64.8 

19.7 

2.8 

11.3 

1.  3 

100.0 
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Air  Force.  Thirty-one  of  the  seventy-eight  respon- 
dents selected  the  bilateral  supplemental  agreement  issued 
under  authority  of  the  Changes  clause  for  a 39.7  percent 
selection  rate.  The  majority  that  chose  this  instrument 
indicated  that  38.7  percent  of  the  time  they  made  changes 
in  specifications  in  the  price  range  of  $10,000  or  less 
(see  Table  13 ) . 

Army . Eighteen  of  the  fifty-four  respondents 
selected  the  bilateral  supple-mental  agreement  issued  under 
the  authority  of  the  Changes  clause  for  a 33.3  percent 
selection  rate.  The  majority  that  chose  this  instrument 
indicated  that  44.4  percent  of  the  time  they  made  changes 
in  specifications  in  the  price  range  of  $10,000  or  less 
through  $100,000  (see  Table  14). 

I 

i 

Navy.  Seventeen  of  the  thirty-five  respondents  j 

selected  the  bilateral  supplem.ental  agreement  issued  under 
authority  of  the  Changes  clause  for  a response  rate  of 
48.6  percent.  The  majority  that  chose  this  instr-ument 
indicated  that  41.2  percent  of  the  time  they  made  specifi- 
cation c.hanges  in  the  over  $10,000  through  $100,000  price 
range  (see  Table  15). 

DLA.  Eleven  of  the  twenty-one  respondents  selected 
bilateral  supplemental  agreement  issued  under  authority  of 
a contract  provision  other  than  the  Changes  clause  for  a 
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TABLE  13 

CONTRACT  MODIFICATION  INSTRUMENT  USED  MOST  FREQUENTLY 

BY  THE  AIR  FORCE 


Specs 

Time/ 

Place 

Perf 

Qty 

2 

0 

0 

6.5 

0. 

0. 

12 

1 

1 

38.7 

3.2 

3.2 

6 

2 

2 

19.  4 

6.5 

6.5 

3 

0 

1 

9.7 

0. 

3.2 

1 

0 

0 

3.2 

0. 

0. 

24 

3 

4 

77.4 

9.7 

12.9 

64 

Count 
Tot  Pet 


No  Cost 


$10, 000 

or 

Less 

Over 
$10, 000 
thru 

$100, 000 

Over 

$100, 000 

thru 

$1,0C-  000 


Over 

$1, 000, 000 


Column 

Total 


Total 


TABLE  15 


CONTRACT  MODIFICATION  INSTRUMENT  USED  MOST  FREQUENTLY 

BY  THE  NAVY 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

Pack/ 

Ship 

.Method 

GFM 

Row 

Total 

$10, 000 
or 

3 

2 

1 

0 

6 

Less 

17.  6 

11.8 

5.  9 

0. 

35.3 

Over 

$10, 000 
thru 

7 

0 

0 

1 

8 

$100, 000 

41.2 

0. 

0. 

5.9 

47.1 

Over 

$100, 000 
thru 

2 

0 

1 

0 

3 

$1,000,000 

11.8 

0. 

5.9 

0. 

17.6 

Column 

12 

2 

2 

1 

17 

Total 

70.6 

11.8 

11.8 

5.9 

100.  0 
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response  rate  of  52.4  percent.  The  majority  that  chose 
this  instrument  indicated  that  45.5  percent  of  the  time 
they  made  changes  in  time/place  of  performance  in  the 
$10,000  or  less  price  range  (see  Table  16). 


Question  10:  Contract  Modification 
Instrument  Preferred  by  Agency 

The  respondents  indicated  that  overall  the  bilateral 
supplemental  agreement  issued  under  authority  of  the  Changes 
clause  was  the  most  preferred  by  their  agencies  indi- 
vidually; the  Air  Force,  Army,  and  Navy  respondents  said 
their  agencies  also  preferred  the  bilateral  supplemental 
agreement  issued  under  the  authority  of  the  Changes  clause. 
As  in  the  question  of  what  contract  modification  instru- 
ment was  used  more  frequently,  the  DLA  respondents  indi- 
cated their  agency  preferred  the  use  of  the  bilateral  sup- 
plemental agreement  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 


All  agencies.  Seventy-eight  of  the  one  hundred 
ninety-four  total  respondents  selected  t.he  bilateral  sup- 
ple.mental  agreement  issued  under  the  authority  of  the 
Changes  clause  for  a 40.2  percent  selection  rate.  The 
m.ajority  that  chose  this  instrument  indicated  that  29.5 
percent  of  the  time  they  made  changes  in  specifications  in 
the  price  range  of  $10,000  or  less  (see  Table  17). 


TABLE  16 


CONTRACT  MODIFICATION  INSTRUMENT  USED  MOST  FREQUENTLY 

BY  THE  DLA 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

1 

Qty 

Row 

Total 

$10, 000 

1 

5 

0 

6 

or 

Less 

9. 1 

45.5 

0. 

54.5 

Over 

$10,000 

1 

1 

3 

5 

thru 

$100, 000 

9.  1 

9.1 

27.3 

45.5 

Column 

2 

6 

3 

11 

Total 

18.2 

54.5 

27.  3 

100.0 

1 
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TABLE  17 


CONTRACT  MODIFICATION  INSTRUMENT  PREFERRED 
BY  ALL  AGENCIES 


Count 
Tot  Pot 


Specs 


Time/ 

Place 

Perf 


Pack/ 

Ship 

Method 


Row 

Total 


Column 


Total  64.1 


16 

20.5 


00 

; 10.3 

2.i 

78 

100.0 


Air  Force.  Thirty-eight  of  the  seventy-eight  Air 


r 


Force  respondents  selected  the  bilateral  supplemental  agree- 
ment issued  under  the  authority  of  the  Changes  clause  for 
a 48.7  percent  selection  rate.  The  majority  that  chose 
this  instrument  indicated  that  39.5  percent  of  the  time 
they  made  changes  in  specifications  in  the  price  range  of 
$10,000  or  less  (see  Table  18). 

Army.  Seventeen  of  the  fifty-four  Army  respondents 
selected  the  bilateral  supplem.ental  agreement  issued  under 
the  authority  of  the  Changes  clause  for  a 31.5  percent 
selection  rate.  The  majority  that  chose  this  instrument 
indicated  that  29.4  percent  of  the  time  they  made  changes 
in  specifications  for  $10,000  or  less  (see  Table  19). 

Navy.  Seventeen  of  the  thirty-five  Navy  respon- 
dents selected  the  bilateral  supplemental  agreement  issued 
under  the  authority  of  the  Changes  clause  for  a 48.6  per- 
cent selection  rate.  The  majority  that  chose  this  instru- 
ment indicated  that  35.3  percent  of  the  time  they  made 
specification  changes  in  the  over  $10,000  through  $100,000 
price  range  (see  Table  20) . 

DLA . Eleven  of  the  twenty-one  DLA  respondents 
\ selected  the  bilateral  supplemental  agreement  issued  under 
authority  of  a contract  provision  other  than  the  Changes 
clause  for  a 52.4  percent  selection  rate.  The  majority 
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TABLE  18 


CONTRACT  MODIFICATION  INSTRUMENT  PREFERRED 
BY  THE  AIR  FORCE 


Count 
Total  Pet 

Specs 

Time/ 

Place 

Perf 

Qtv 

gfm 

Row 

Total 

2 

0 

0 

. 0 

2 . . 

: 

No  Cost 

5.  3 

0. 

0. 

0. 

5.  3 

$10,000 

15 

2 

1 

0 

18 

or  Less 

39.  5 

5.3 

2.6 

0. 

47.  4 

Over 

$10,000 

thru 

7 

2 

2 

1 

12 

$100,000 

18.4 

5.  3 

5.  3 

2.6 

31.  6 

Over 
$100, 000 
thru 

4 

0 

1 

0 

5 

$1, 000,000 

10.5 

0. 

2.6 

0. 

13.  2 

) 

1 

0 

0 

0 

1 

Over 

$1,000, 000 

2.6 

0. 

0. 

0. 

2.6 

I 

' Column 

1 

29 

4 

4 

1 

38 

[ Total 

[ 

76.  3 

10.5 

10.5 

2.6 

100.  0 

TABLE  19 


CONTRACT  MODIFICATION  INSTRUMENT  PREFERRED 
BY  THE  ARMY 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

Qty 

Row 

Total 

1 

1 

0 

2 

No  Cost 

5.  9 

0. 

11.  8 

$10,000 

5 

3 

3 

11 

or 

Less 

29.  4 

17.6 

17.6 

64.7 

Over 

$10,000 

2 

2 

0 

4 

thru 

$100,000 

11.8 

11.8 

1 

0. 

23.5 

Column 

8 

6 

3 

17 

Total 

47.1 

35.3 

17.6 

100.0 

TABLE  20 


CONTRACT  MODIFICATION  INSTRUMENT  PREFERRED 
BY  THE  NAVY 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

Pack/ 

Ship 

Method 

GFM 

Row 

Total 

$100, 000 
or 

2 

2 

1 

0 

5 

Less 

11.8 

11.8 

5.9 

0. 

29.4 

Over 

$10,000 

thru 

6 

0 

0 

1 

7 

$100, 000 

35.3 

0. 

0. 

5.  9 

41.2 

Over 

$100,000 

thru 

4 

0 

1 

0 

5 

$1,000,000 

23.5 

0. 

5.9 

0. 

29.4 

Column 

12 

2 

2 

1 

17 

Total 

70.  6 

11.8 

11.3 

5.9 

100.0 

that  chose  this  instrument  indicated  that  45.5  percent  of 
the  time  they  made  changes  in  time/place  of  performance  in 
the  $10,000  or  less  price  range  (see  Table  21). 

Question  11:  Contract  Modification 
Instrument  Preferred  by  the  Individual 

The  respondents  indicated  an  overall  preference 
for  the  bilateral  supplemental  agreement  issued  under  the 
authority  of  the  Changes  clause.  Respondents  from  the  Air 
Force,  Army,  and  Navy  individually  also  preferred  to  use 
the  bilateral  supplemental  agreement  issued  under  the 
authority  of  the  Changes  clause.  The  DLA  respondents 
again  selected  the  bilateral  supplement  agreement  issued 
under  authority  of  a contract  provision  other  than  the 
Changes  clause  as  the  preferred  instrument. 

All  Agencies.  Seventy  of  the  one  hundred  ninety- 
four  respondents  selected  the  bilateral  supplemental  agree- 
ment issued  under  the  authority  of  the  Changes  clause  for 
a 36.1  percent  selection  rate.  The  majority  that  pre- 
ferred this  instrument  indicated  that  they  made  changes  in 
specifications  i.n  the  $10,  000  or  less  price  range  28.6  per- 
cent of  the  time  (see  Table  22) . 

Air  Force.  Thirty-three  of  the  seventy-eight  Air 
Force  respondents  selected  the  bilateral  supplement  agree- 
ment issued  under  the  authority  of  the  Changes  clause  for 
a 42.3  percent  selection  rate.  The  majority  that  preferred 
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TABLE  21 

CONTRACT  MODIFICATION  INSTRUMENT  PREFERRED 
BY  THE  DLA 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

Qty 

r ■ ■■ 

Row 

Total 

$10,000 

1 

5 

0 

6 

or 

Less 

9.1 

45.  5 

0. 

54.5 

Over 

$10, 000 

1 

1 

3 

5 

thru 

$100,000 

9.  1 

9.1 

27.3 

45.  5 

Column 

2 

6 

3 

11 

] 


Total 


18.  2 


54.  5 


27.  3 


100.0 


TABLE  22 


CONTRACT  MODIFICATION  INSTRUMENT  PREFERRED 
BY  ALL  RESPONDENTS 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

Pack/ 

Ship 

Method 

Qty 

GFM 

Row 
To  tal 

3 

1 

1 

0 

1 

6 

No  Cost 

4.  3 

1.  4 

1.  4 

0. 

1.4 

8.6 

$10,000 

20 

9 

1 

3 

0 

33 

or 

Less 

28.6 

12.9 

1.4 

4.3 

0. 

47.1 

Over 

17 

$10,000 

2 

0 

1 

2 

22 

thru 

$100, 000 

2 3.4 

2.9 

i 

0. 

1 

1.  4 

1 

2.9 

31.4 

Over 

j 

$100, 000 

6 

0 

1 

1 

0 

8 

thru 

$1, 000, 000 

8.6 

1 

0. 

1.  4 

1.4 

0. 

11.  4 

Over 

$1, 000,000 

1 

0 

0 

0 

0 

1 

1.4 

0. 

0. 

0. 

0. 

1.4 

Column 

47 

12 

3 

5 

3 

70 

Total 

67.  1 

17.1 

4.3 

7.1 

4.3 

100.  0 

this  instrument  indicated  that  they  made  changes  in  spe- 
cifications in  the  $10,000  or  less  price  range  36.4  percent 
of  the  time  (see  Table  23). 

Army.  Fifteen  of  the  fifty-four  Army  respondents 
selected  the  bilateral  supplemental  agreement  issued  under 
the  authority  of  the  Changes  clause  as  the  most  preferred 
for  a 27.8  percent  selection  rate.  The  majority  of  the 
respondents  that  selected  this  instrument  indicated  that 
they  made  specifications  changes  in  the  $10, 000  or  less 
through  the  $100,000  price  range  53.4  percent  of  the  time 
(see  Table  24) . 

Navy.  Sixteen  of  the  thirty-five  Navy  respondents 
indicated  a 45.7  percent  preference  for  the  bilateral 
supplemental  agreement  issued  under  the  authority  of  the 
Changes  clause.  The  majority  of  the  Navy  respondents 
selecting  this  instrument  indicated  they  made  specifica- 
tions changes  in  the  over  $10,000  through  $100,000  price 
range  37.5  percent  of  the  time  (see  Table  25). 

DLA . Eight  of  the  twenty-one  DLA  respondents 
selected  the  bilateral  supplemental  agreement  issued  under 

1 

' authority  of  a contract  provision  other  than  the  Changes 

clause  for  a preference  rate  of  38.1  percent.  The  majority 
of  the  DLA  respondents  selecting  this  instrument  indicated 
they  made  changes  in  time/place  of  performance  in  the 
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TABLE  23 


CONTRACT  MODIFICATION  INSTRUMENT  PREFERRED 
BY  AIR  FORCE  RESPONDENTS 


Count 
Total  Pot 

Specs 

Time/ 

Place 

Perf 

Pack/ 

Ship 

Method 

Qty 

GFM 

Row 

Total 

2 

0 

1 

0 

1 

4 

No  Cost 

6.1 

0. 

3.0 

0. 

3.0 

12.1 

$10, 000 
or 

12 

1 

0 

1 

0 

14 

Less 

36.4 

3.0 

0. 

3.0 

0. 

42.4 

Over 

$10,000 

thru 

7 

1 

0 

1 

1 

10 

$100, 000 

21.2 

3.  0 

0. 

3.0 

1 

3.0 

30.  3 

Over 

] 

$100, 000 
thru 

3 

0 

0 

1 

0 

4 

$1,000,000 

9.1 

0. 

0. 

3.0 

0. 

12.1 

Over 

1 

0 

0 

0 

0 

1 

$1,000,000 

3.0 

i 

0. 

0. 

0. 

0. 

3.0 

Column 

25 

2 

1 

3 

2 

33 

Total 

75.  8 

6.  1 

3.  0 

9.  1 

6.  1 

100. 0 
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NATIONAL  BUREAU  OF  STANDARDS 

MICROCOPY  RESOLUTION  TEST  CHART 


$10, 000 

or 

Less 

Over 

$10,000 

thru 

$100,000 

Over 

$100,000 

thru 

$1, 000, 000 


3 

1 

18.8 

6.3 

6 

0 

37.  5 

0. 

3 

0 

18.8 

0. 

12 

1 

75.  0 

6.3 

2 

12.5 


5 

31.  3 
7 

43.8 

4 

25.0 


.6 


100.  0 


$10,000  or  less  price  range  50.0  percent  of  the  time  (see 
Table  26). 


Statistical  Tests 

The  data  for  the  statistical  tests,  obtained  from 
questions  12  through  15  are  presented  in  Tables  27  through 
36.  The  variables  are  portrayed  on  the  table  rows  and  are 
labeled  accordingly.  The  six  contract  modification  instru- 
ments being  ranked  are  portrayed  on  the  table  columns  and 
labeled  1 through  6.  Column  1 represents  a unilateral 
change  order  issued  under  authority  of  the  Changes  clause. 
Column  2 represents  a bilateral  supplemental  agreement 
issued  under  authority  of  the  Changes  clause.  Column  3 
represents  a unilateral  change  order  issued  under  authority 
of  a contract  provision  other  than  the  Changes  clause. 
Column  4 represents  a bilateral  supplemental  agreement 
issued  under  authority  of  a contract  provision  other  than 
the  Changes  clause.  Column  5 represents  a bilateral  sup- 
plemental agreement  for  performance  outside  the  scope  of 
the  contract.  Column  6 represents  a constructive  change 
order  based  on  some  informal  action  on  the  part  of  the 
government  that  implies  a contract  is  to  be  modified. 

Test  I;  Friedman  Test 

This  test  was  conducted  on  the  data  collected  from 
all  the  agencies  and  then  individually  on  the  data  received 
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TABLE  26 


CONTRACT  MODIFICATION  INSTRUMENT  PREFERRED 
BY  DLA  RESPONDENTS 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

Qty 

Row 

Total 

$10,000 

0 

4 

0 

4 

or 

Less 

0. 

50.  0 

0. 

50.  0 

Over 

$10,000 

1 

0 

3 

4 

thru 

$100, 000 

12.  5 

0. 

37.5 

50.  0 

Column 

1 

4 

3 

8 

Total 


12.5 


50.  0 


37.5 


100.  0 


from  the  Air  Force,  Army,  Navy,  and  DLA.  The  hypotheses 
■were  the  same  in  each  instance. 

Hq:  The  contract  modification  instruments  are 
independent  of  the  variables; 

The  contract  modification  instruments  are 
dependent  on  the  variables. 

The  test  statistic  (see  Tables  27  through  31)  was: 

= SiTT^  - 3N0C+1) 


For  the  overall  data,  the  computed  chi-square  value  was 
16.286  (see  Table  27).  The  chi-square  critical  value  for 
(k-1),  where  k = number  of  columns,  or  5 degrees  of  freedom, 
and  an  alpha  of  .05,  was  11.070.  The  test  statistic, 

16.286,  was  greater  than  11.070;  therefore,  the  null  hypo- 
thesis was  rejected  and  the  alternate  hypothesis  accepted. 
The  contract  modification  instruments  are  dependent  on  the 
variables. 

The  test  conducted  on  the  Air  Force  data  yielded 
similar  results.  The  computed  chi-square  value  was  14.714 
and  the  chi-square  critical  value  was  11.070  (see  Table 
28).  Since  14.714  was  greater  than  11.070,  the  null  hypo- 
thesis was  rejected  and  the  alternate  hypothesis  accepted. 
Contract  modification  instrxaments  are  dependent  on  the 
variables;  flexibility,  manageability,  cost  effectiveness, 
and  timeliness. 


TABLE  27 


RANKS  ASSIGNED  TO  CONTRACT  MODIFICATION  INSTRUMENTS 
(FRIEDMAN  TEST:  ALL  AGENCIES) 


1 

2 

3 

4 

5 

6 

Flexible 

2 

1 

4 

3 

5 

6 

manageable 

3 

1 

4 

2 

5 

6 

Cost 

Effective 

3 

1 

4 

2 

5 

6 

Timely 

1 

3 

2 

4 

6 

5 

R. 

3 

9 

6 

14 

11 

21 

23 

«/) 

81 

36 

196 

121 

441 

529 

2 

^ (Rj  )=  1401  a = .05,  d.f.  = 5 

NK(k+l)  =168  k = 6 

3N(k+l)  =84  N = 4 


The  remaining  tests  indicated  consistency  among 
the  various  DOD  procuring  activities.  The  computed  chi- 
square  values  for  the  Army,  Navy,  and  DLA  were  13.286, 
15.286,  and  17.571,  respectively  (see  Tables  29  through  31). 
The  chi-square  critical  value  remained  the  same,  11.070,  as 
the  degrees  of  freedom  and  alpha  level  did  not  change. 
Consequently,  the  null  hypothesis,  that  contract  modifica- 
tion instruments  are  independent  of  the  variables,  was 
rejected  in  every  test. 

Results  of  the  Friedman  Test  indicated  that  choice 
of  a contract  modification  instrument,  be  it  a unilateral 
change  order  issued  pursuant  to  the  Changes  clause,  or  a 
bilateral  supplemental  agreement  issued  pursuant  to  a 
clause  other  than  the  Changes  clause,  or  any  of  the  other 
instruments  described  in  the  survey,  is  dependent  on  the 
variables  of  flexibility,  manageability,  cost  effectiveness, 
and  timeliness. 

Test  II:  Kendall  Test 

The  Kendall  Test  was  also  conducted  on  the  overall 
data  and  then  individually  on  the  Air  Force,  Army,  Navy, 
and  DLA  data.  The  hypothesis  test  conducted  was: 

Hq:  There  is  independence  among  the  variables; 

There  is  no  independence  among  the  variables. 

The  test  statistic  (see  Tables  32  through  36)  was: 
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TABLE  29 


RANKS  ASSIGNED  TO  CONTRACT  MODIFICATION  INSTRUMENTS 

(FRIEDMAN  TEST:  ARMY) 


\lnstnmaits 

Variables 

1 

1 

2 

3 

4 

5 

6 

Flexible 

1 

5 

2 

3 

1 ' 

4 

6 

Itenageable 

3 

1 

4 

2 

5 

6 

Cost 

Effective 

1 

2 

4 

3 

5 

6 

Tirtely 

1 

3 

2 

4 

6 

5 

R. 

D 

10 

8 

13 

10 

20 

23 

100 

64 

169 

100 

400 

529 

T.  (Rj^)  = 1362 
Nk(k+1)  = 168 

= 84 

Xr~  = 13.286 


3N(k+l) 
2 


a = .05,  d.f.  = 5 
k = 6 
N = 4 

, = 11.070 
■'•critical 
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TABLE  30 


RANKS  ASSIGNED  TO  CONTRACT  MODIFICATION  INSTRUMENTS 
(FRIEDMAN  TEST:  NAVY) 


N^Instruments 

VariablesNv 

1 

2 

3 

i 

i 

4 

5 

6 

Flexible 

-J 

1 

4 

5 

6 

Manageable 

3 

1 

5 

2 

4 

6 

Cost 

Effective 

3 

1 

4 

2 

5 

6 

Timely 

1 

3 

2 

4 

6 

5 

R. 

3 

1 

10 

6 

15 

10 

20 

i 

23 

100 

36 

225 

100 

1 

1 

400 

529 

Z (Rj^)  = 1390  a = .05,  d.f.  = 5 

Nlc(k+1)  =168  k = 6 

3N(k+l)  =84  N = 4 

Xr^  - 15.286  = 11.070 
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TABLE  31 


RANKS  ASSIGNED  TO  CONTRACT  MODIFICATION  INSTRUMENTS 
(FRIEDMAN  TEST:  DLA) 


5 

6 

5 

6 

5 

6 

5 

6 

5 

6 

20 

24 

400 

576 

r (Rj  ) = 1422 
Nk(lc+1)  = 168 


a = . 05,  d.f . = 5 
k = 6 


3N(k+l)  = 84 


N = 4 


Xr^  = 15.571 


- = 11.070 
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Table  32  shows  that  the  computed  s = 223  for  the  overall 
data.  s . . , for  an  alpha  of  .05,  a k of  4,  where  k 

equals  the  number  of  variables,  and  an  N of  6,  where  N is 
the  number  of  instruments  being  ranked,  was  143.3.  Since 
s = 228  was  greater  than  143.3,  the  null  hypothesis  was 
rejected  and  the  alternate  hypothesis  accepted:  There  is 
no  independence  among  the  variables.  The  Kendall  Coeffi- 
cient of  Concordance:  W was  computed  by  the  following: 

s 

W = 

l/12k^ (N^-N) 

The  value  of  W indicated  the  degree  of  association  among 
the  variables.  Computed  from  the  data  for  all  of  the  DOD 
procurement  activities,  its  value  was  .814.  Since  W can 
only  take  on  values  from  0 to  +1,  the  closer  W is  to  1,  the 
greater  the  amount  of  association  among  the  variables,  which 
in  this  case  were  flexibility,  manageability,  cost  effec- 
tiveness, and  timeliness. 

The  Kendall  Test  performed  on  the  Air  Force  data 
resulted  in  a computed  s of  206.  Again  the  computed  s, 

206,  was  greater  than  ^^ritical'  null  hypo- 

thesis was  rejected,  and  the  alternate  hypothesis  accepted. 
There  is  no  independence,  or  restated,  there  is  dependence 
among  the  variables.  The  Kendall  Coefficient  of  Concordance 
W for  the  Air  Force  data  was  .736,  still  a large  degree  of 
association  among  the  variables  (see  Table  33) . 
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TABLE  33 


RANKS  ASSIGNED  TO  CONTRACT  MODIFICATION  INSTRUMENTS 
(KENDALL  TEST:  AIR  FORCE) 


\ Instruments 


I 

5 

1 

6 

j 

5 

6 

5 

6 

4 

6 

6 

5 

20 

1 

23 

i 

6 

i 

9 

36  ' 

1 

81 

Mean  R^  = = 14 


lyl2k^(N^-N)  = 280 


a = .05 
k = 4 
N = 6 

^critical  = 
W = .736 
92 


As  with  the  Friedman  Test,  the  Kendall  Test  pro- 
duced similar  results  on  the  Army,  Navy,  and  DLA  data  as 
was  experienced  with  the  Air  Force  and  overall  data.  The 
only  result  that  changed  was  the  value  of  W.  The  computed 
s for  the  Army,  Navy,  and  DLA  were  186  {see  Table  34), 

214  (see  Table  35),  and  246  (see  Table  36),  respectively. 

For  each  agency,  the  computed  s was  greater  than  ^c;j-itical 
which  remained  143.3  for  each  test.  The  null  hypothesis 
was  rejected  and  the  alternate  hypothesis  accepted; 
dependency  exists  among  the  variables  of  flexibility, 
manageability,  cost  effectiveness,  and  timeliness.  The 
Kendall  Coefficient  of  Concordance;  W took  on  values  of  .664 
for  the  Army  (see  Table  34),  .764  for  the  Navy  (see  Table 
35),  and  .879  for  the  DLA  (see  Table  36)  data.  Again  these 
numbers  demonstrated  a relatively  high  amount  of  associa- 
tion ami'^ng  the  variables. 

Results  of  Actual  Contract  Modification 
Data  Collection 

In  order  to  determine  if  the  data  collected  by 
sample  reflected  actual  conditions  in  the  field,  75  actual 
contract  modification  documents  were  examined.  Contract 
files  at  the  Procurement  Division,  2750th  Air  Base  Wing, 
Wright-Patterson  AFB,  Ohio,  the  Defense  Electronic  Suppl:- 
Center  (DESC) , Dayton,  Ohio,  and  the  Defense  Contract 
Administration  Management  Area  (DCASMA) , Dayton,  Ohio, 
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TABLE  34 


RANKS  ASSIGNED  TO  CONTRACT  MODIFICATION  INSTRUMENTS 
(KENDALL  TEST;  ARMY) 


TABLE  36 


RANKS  ASSIGNED  TO  CONTRACT  MODIFICATION  INSTRUMENTS 
(KENDALL  TEST:  DLA) 

vinstrumentd  j [ I I I 


Variable^ 

Flexible 

Manag^ble 


Cost 

Effective 


Timely 


R'  - M 
J N 


(;-7J 


2 

5 

6 

2 

5 

1 

1 

6 

2 

5 

6 

2 

5 

6 

8 

20 

24 

-6 

6 

10 

36 

36 

.100 

ER. 

Mean  R.  = = 14 

J N 

/ v2 


s = Z 


h - 


l/12k^(N^-N)  = 280 


a = .05 
k = 4 
N = 6 

s ...  , = 143.3 

critical 

W = .879 
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provided  the  data.  The  data  are  presented  in  Tables  37 
through  40. 

No  attempt  was  made  to  statistically  correlate  the 
actual  contract  modification  data  (Tables  37  through  40) 
with  the  survey  data.  However,  the  actual  data  compared 
favorably  with  the  data  obtained  by  survey.  For  example, 
there  was  a one-to-one  correlation  between  respondent  data 
and  the  actual  contract  file  data  for  all  six  modification 
instruments  and  for  all  five  reasons  for  modifying  con- 
tracts. These  findings  tended  to  support  the  validity  of 
the  survey  questionnaire  data. 

Joint  Frequency  Distributions 

Joint  frequency  distributions  were  generated  on  the 
actual  observed  contract  modifications  as  well  as  the  sur- 
vey data.  Contract  modifications  were  reviewed  from  the 
Air  Force,  Navy,  and  DLA  agencies.  The  overall  cross- 
tabulation tables  indicated  that  the  bilateral  supplemental 
agreement  issued  under  the  authority  of  the  Changes  clause 
was  used  most  frequently.  An  overall  table  is  presented 
for  this  instrument. 

Individual  tables  are  presented  only  for  the  Air 
Force  and  the  DLA.  The  limited  number  of  observations 
from  the  Navy  contract  modifications  renders  separate 
analysis  meaningless. 
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Twenty-nine  of  the  seventy-five  contract  modifica- 


'I 


tions  had  changes  made  using  a bilateral  supplemental  agree- 
ment issued  under  the  authority  of  the  Changes  clause  for  a 
rate  of  38.7  percent.  There  were  34.5  percent  of  these 
changes  made  for  specifications  changes  in  the  $10,000  or 
less  price  range  (see  Table  41) . 

Twenty-seven  of  the  forty  Air  Force  contract  modi- 
fications had  changes  made  using  a bilateral  supplemental 
agreement  issued  under  the  authority  of  the  Changes  clause 
for  a rate  of  67.5  percent.  There  were  37.0  percent  of 
these  changes  made  for  specifications  changes  in  the  $10,000 
or  less  price  range  (see  Table  42) . 

Seventeen  of  the  twenty-seven  DLA  contract  modifi- 
cations had  changes  made  using  a bilateral  supplemental 
agreement  issued  under  authority  of  a contract  provision 
other  than  the  Changes  clause  for  a rate  of  63.0  percent. 
There  were  41.2  percent  of  these  changes  made  for  time/ 
place  performance  modifications  in  the  price  range  of 
$10,000  or  less  (see  Table  43). 


100 


TABLE  41 

CONTRACT  MODIFICATION  INSTRUMENT  USED  MOST  FREQUENTLY: 

ALL  ACTUAL  DATA 


Time/ 

Pack/ 

Count 

Place 

Ship 

Row 

Tot  Pet 

Specs 

Perf 

Method 

Qty 

Total 

1 

■1 

0 

1 

9 

No  Cost 

24.1 

B 

0. 

3.4 

31.0 

$10,000 

10 

3 

1 

■1 

15 

or 

Less 

34.5 

10.  3 

3.4 

B 

51.7 

Over 

$10,000 

■1 

0 

2 

4 

through 

$100,000 

Hi 

■ 

0. 

6.9 

13.8 

Over 

$100,000 

0 

0 

0 

1 

1 

through 
$1,  000,000  j 

0. 

0. 

0. 

3.4 

3.4 

Column 

5 

1 

5 

29 

Total 

1 

17.2 

3.4 

17.2 

100.  0 

TABLE  42 


CONTRACT  MODIFICATION  INSTRUMENT  USED  MOST  FREQUENTLY: 
AIR  FORCE  ACTUAL  DATA 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

Pack/ 

Ship 

Method 

Qty 

Row 

Total 

6 

1 

0 

1 

8 

No  Cost 

22.2 

3.  7 

0. 

3.7 

3.7 

$10, 000 

10 

3 

1 

1 

15 

or 

Less 

37.0 

11.1 

3.7 

3.7 

55.6 

Over 
$10, 000 

1 

1 

0 

2 

4 

through 

$100,000 

3.7 

3.7 

0. 

7.4 

14.8 

Column 

17 

5 

1 

4 

27 

Total 

63.0 

18.  5 

3.7 

14.8 

100.  0 

TABLE  43 


CONTRACT  MODIFICATION  INSTRUMENT  USED  MOST  FREQUENTLY; 

DLA  ACTUAL  DATA 


Count 
Tot  Pet 

Specs 

Time/ 

Place 

Perf 

GFM 

Row 

Total 

2 

5 

8 

7 

No  Cost 

11.  8 

29.  4 

0. 

41.  2 

$10, 000 

2 

7 

1 

10 

or 

Less 

11.  8 

41.  2 

5.9 

58.8 

Column 

4 

12 

1 

17 

Total 

23.  5 

70.  6 

5.  9 

100.  0 

CHAPTER  IV 


RESEARCH  FINDINGS  AND  RECOMMENDATIONS 


Introduction 

Chapter  IV  contains  the  findings  for  each  research 
question,  corollary  findings,  and  recommendations  for 
further  research.  Research  question  findings  are  presented 
for  all  Federal  agencies  as  well  as  the  Air  Force,  Army, 
Navy  and  DLA. 


Research  Questions 
Research  Question  One 

What  available  contract  modification  instruments 
are  preferred  by  the  different  DOD  contract  placement  and 
management  agencies? 

Questionnaire  question  number  10  asked  each  respon- 
dent to  indicate  which  contract  modification  instr’oment 
his  agency  preferred  him  to  use.  This  question  was  intended 
to  be  used  to  answer  research  question  one.  In  order  to 
stipulate  a specific  situation  when  using  a contract  modifi- 
cation instrument,  a joint  frequency  distribution  was  per- 
formed on  the  question.  The  two  factors  stipulating  the 
situation  were  price  range  and  reason  for  the  contract 
modification. 


104 


Air  Force.  The  majority  of  the  Air  Force  respon- 
dents indicated  their  agency  preferred  them  to  use  the 
bilateral  supplemental  agreement  issued  under  the  authority 
of  the  Changes  clause.  Joint  frequency  distribution  analy- 
sis indicated  that  this  instrument  was  to  be  used  when 
making  changes  in  the  specifications  in  the  price  range  of 
$10,000  or  less . 

Army.  The  majority  of  the  Army  respondents  indi- 
cated their  agency  preferred  them  to  use  the  bilateral 
supplemental  agreement  issued  under  the  authority  of  the 
Changes  clause.  Joint  frequency  distribution  analysis 
indicated  that  the  instrument  was  to  be  used  when  making 
changes  in  specifications  in  the  price  range  of  $10,000  or 
less . 

Navy.  The  majority  of  the  Navy  respondents  indi- 
cated their  agency  preferred  them  to  use  the  bilateral 
supplemental  agreement  issued  under  the  authority  of  the 
Changes  clause.  Joint  frequency  distribution  analysis 
indicated  that  this  instrument  was  to  be  used  when  making 
changes  in  specifications  in  the  over  $10,000  through 
$100,000  price  range. 

DLA.  The  majority  of  the  DLA  respondents  indicated 
their  agency  preferred  them  to  use  the  bilateral  supple- 
mental  agreement  issued  under  authority  of  a contract 
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provision  other  than  the  Changes  clause.  Joint  frequency 
distribution  analysis  indicated  that  this  instrument  was 
to  be  used  when  making  changes  in  the  time  and/or  place  of 
performance  in  the  price  range  of  $10,000  or  less. 

All  Agencies.  The  overall  majority  of  the  respon- 
dents from  all  DOD  agencies  indicated  a DOD  preference  for 
the  bilateral  supplemental  agreement  issued  under  the 
authority  of  the  Changes  clause.  Joint  frequency  distribu- 
tion analysis  indicated  that  this  instrument  was  used  when 
making  changes  in  specifications  in  the  price  range  of 
$10,000  or  less. 

Supportive  Data.  In  order  to  substantiate  the 
surveyed  responses  to  question  10,  questions  9 and  11  were 
included  in  the  questionnaire.  Question  9 asked  the  respon- 
dent to  indicate  the  contract  modification  instrument  he 
uses  most  frequently.  Question  11  asked  the  respondent  to 
indicate  the  contract  modification  instrument  he  prefers 
to  use.  The  researchers  felt  the  responses  to  these  two 
questions  should  be  essentially  the  same  as  the  responses  to 
question  10  if  the  data  was  to  be  regarded  as  factual.  For 
all  agencies,  the  majority  of  the  respondents  responded  the 
same  way  to  questions  9 and  11  as  they  had  to  question  10. 

The  accumulation  of  actual  observed  contract  modi- 
fication data  was  also  accomplished  to  lend  support  for 
question  10  responses.  Here  again,  the  data  accumulated 
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supported  the  responses  given  in  the  survey.  The  majority 
of  the  actual  Air  Force  contract  modifications  observed 
indicated  the  use  of  the  bilateral  supplemental  agreement 
issued  under  the  authority  of  the  Changes  clause  in  situa- 
tions where  changes  in  specifications  in  the  price  range  of 
$10,000  or  less  were  made.  The  majority  of  the  actual  DLA 
contract  modifications  observed  indicated  the  use  of  the 
bilateral  supplemental  agreement  issued  under  authority  of 
a contract  provision  other  than  the  Changes  clause  where 
changes  in  time  and/or  place  of  performance  in  the  price 
range  of  $10,000  or  less  were  made.  The  overall  (including 
Air  Force,  Navy,  and  DLA)  actual  contract  modification  data 
indicated  the  use  of  the  bilateral  supplemental  agreement 
issued  under  the  authority  of  the  Changes  clause  where 
changes  in  specifications  in  the  price  range  of  $10,000 
or  less  were  made. 


Conclusions.  The  research  indicated  that  there  is 
a definite  preference  for  a particular  contract  modifica- 
tion instrument  among  the  various  DOD  agencies.  While 
survey  respondents  indicated  a range  of  preferences,  the 
majority  indicated  the  bilateral  supplemental  agreement 
issued  under  the  authority  of  the  Changes  clause.  The  only 
exception  to  this  was  the  DLA  respondents  where  the  bilateral 
supplemental  agreement  issued  under  authority  of  a contract 
ision  other  than  the  Changes  clause  was  indicated  as  the 
preferred  agency  contract  modification  instrument. 


\ 
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Research  Question  Two 

What  is  the  overall  efficiency  of  these  instruments 
in  terms  of  flexibility,  manageability,  cost  effectiveness, 
and  timeliness? 

Research  question  two  findings  were  based  on  the 
statistical  analysis  of  questions  12  through  15  of  the 
questionnaire.  Respondents  were  instructed  to  base  their 
responses  to  questions  12  through  15  on  the  price  range 
and  type  of  contract  modifications  that  they  most  frequently 
accomplished. 

All  Agencies.  Respondents  from  all  agencies  ranked 
a bilateral  supplemental  agreement  issued  under  the  author- 
ity of  the  Changes  clause  as  the  most  flexible,  manageable, 
and  cost  effective  contract  modification  instrument.  A 
unilateral  change  order  issued  under  the  authority  of  the 
Changes  clause  was  considered  the  most  timely  contract 
modification  instrument.  The  overall  most  efficient  con- 
tract modification  instrument  was  a bilateral  supplemental 
agreement  issued  under  authority  of  the  Changes  clause. 


Air  Force.  Air  Force  respondents  indicated  that  a 
unilateral  change  order  issued  under  authority  of  the 
Changes  clause  was  the  most  flexible  and  timely  contract 
modification  instrument.  A bilateral  supplemental  agreement 
issued  under  authority  of  the  Changes  clause  was  considered 
most  manageable  and  cost  effective.  The  overall  most  efficient 
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contract  modification  instrument  selected  was  the  bilateral 


supplement  agreement  issued  under  the  authority  of  the 
Changes  clause. 

Army.  Army  respondents  ranked  a bilateral  supple- 
mental agreement  issued  under  a contract  provision  other 
than  the  Changes  clause  as  the  most  flexible  contract 
modification  instrument.  A bilateral  supplemental  agree- 
ment issued  under  authority  of  the  Changes  clause  was  con- 
sidered most  manageable.  A unilateral  change  order  issued 
under  authority  of  the  Changes  clause  was  considered  most 
cost  effective  and  timely.  The  overall  most  efficient  con- 
tract modification  instrument  was  a bilateral  supplemental 
agreement  issued  under  authority  of  the  Changes  clause. 

Navy.  Navy  respondents  ranked  a bilateral  supple- 
mental agreement  issued  under  authority  of  the  Changes 
clause  as  the  most  flexible,  manageable,  and  cost  effec- 
tive. A unilateral  change  order  issued  under  authority  of 
the  Changes  clause  was  considered  the  most  timely  instru- 
ment. The  overall  most  efficient  contract  modification 
instrument  was  a bilateral  supplemental  agreement  issued 
under  authority  of  the  Changes  clause. 

DLA.  DLA  respondents  ranked  a unilateral  change 
order  issued  under  authority  of  the  Changes  clause  as  the 
most  flexible,  cost  effective  and  timely  contract 


109 


modification  instrument.  A bilateral  supplemental  agree- 
ment issued  under  authority  of  the  Changes  clause  was  con- 
sidered the  most  manageable.  The  overall  most  efficient 
contract  modification  instrument  was  the  bilateral  supple- 
mental  agreement  issued  under  the  authority  of  the  Changes 


clause. 


Conclusions.  A bilateral  supplemental  agreement 
issued  under  the  authority  of  the  Changes  clause  was 
selected  as  the  most  efficient  contract  modification  instru- 
ment. While  there  was  some  difference  among  the  agencies 
as  to  which  contract  modification  instrument  is  most 
flexible,  manageable,  and  cost  effective,  all  responded 
that  the  unilateral  change  order  issued  pursuant  to  the 
Changes  clause  was  the  most  timely. 


Corollary  Findings 


In  addition  to  iifformation  gleaned  from  the  data 
used  to  answer  the  research  questions,  other  significant 
information  became  apparent  once  the  survey  data  was 
examined.  These  corollary  findings  are  presented  in  this 
section  as  additional  points  of  interest. 


Finding  One 

The  course  prerequisites  for  the  courses  surveyed 
are  very  specific  as  to  procurement  experience  required 
before  attending  the  continuing  education  courses 
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(see  Figure  1,  page  27) . However,  59  of  the  253  respon- 
dents who  returned  the  questionnaires  indicated  that  they 
had  no  procurement  related  experience. 


It  would  appear  that  23.32  percent  of  the  personnel 
sent  to  the  procurement  courses  by  the  various  DOD  agencies 
are  not  qualified  to  attend.  If  this  is  true,  then  either 
23.32  percent  of  the  qualified  procurement  personnel  are 
not  receiving  the  training  required,  or  the  continuing 
education  courses  are  being  maintained  at  too  great  a 
capacity. 

Finding  Two 

The  procurement  career  field  appears  to  contain  a 
large  number  of  people  that  have  cross-trained  from  other 
career  fields.  The  survey  indicated  that  30.9  percent  of 
the  respondents  had  over  16  years  federal  service  time, 

■ but  only  11.9  percent  had  over  16  years  procurement  experi- 
ence. Accordingly,  40.7  percent  of  the  respondents  indi- 
cated they  had  one  through  three  years  experience,  but 
only  20.6  percent  of  the  respondents  indicated  only  one 
through  three  years  federal  service. 

This  could  indicate  an  aging  procurement  work 

■ force.  The  DOD  could  help  alleviate  this  attrition  exodus 

by  obtaining  and  developing  entry  level  personnel  for  a 

I 

* full  procurement  service  career. 

i 

I 

I 

I 

I 111 

I 

i 

1 


1 


1 


Finding  Three 

Contract  modifications  are  being  accomplished  by 
DOD  agencies  for  significant  dollar  amounts.  Respondents 
from  all  the  agencies  indicated  that  48.4  percent  of  the 
time  they  made  modifications  in  the  price  range  of  $10,000 
through  $1,000,000.  Conversely,  only  8.8  percent  of  the 
respondents  indicated  they  normally  made  no  cost  contract 
modifications.  This  dollar  range  represents  a potential 
for  a large  amount  of  contract  cost  growth.  Added  emphasis 
on  contract  modification  control  could  help  reduce  contract 
cost  growth. 

Finding  Four 

The  majority  of  the  respondents  indicated  that  the 
reason  they  made  contract  modifications  was  for  changes  in 
specifications.  This  one  reason  represented  53.6  percent 
of  the  contract  modification  actions.  It  would  appear 
there  is  a problem  in  either  adequate  initial  design  defini- 
tion or  in  making  design  changes  based  on  revised  require- 
ments . 

Finding  Five 

There  appears  to  be  a lack  of  understanding  among 
DOD  procurement  personnel  concerning  the  scope  of  the 
Changes  clause.  While  81.5  percent  of  the  respondents 
indicated  they  made  contract  modifications  for  reasons 
covered  by  a Changes  clause,  only  51.5  percent  indicated 

112 


J 


they  issued  either  unilateral  or  bilateral  changes  under 
the  authority  of  the  Changes  clause. 


i 
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This  finding  indicates  a need  for  increased 
emphasis  on  correct  contract  modification  procedures.  The 
greater  emphasis  could  come  from  formal  procurement  educa- 
tion. 

Finding  Six 

A significant  portion  of  the  respondents  indicated 
that  they  did  not  know  which  contract  modification  instru- 
ment their  agency  preferred.  If  this  is  true,  it  could 
indicate  a lack  of  policy  guidance  from  the  DOD  agencies. 
Overall,  some  20.1  percent  of  the  respondents  indicated 
they  did  not  know  which  instrument  their  agency  preferred. 
Respondents  from  the  Army  selected  the  do  not  know  response 
31.5  percent  of  the  time.  Conversely,  DLA  respondents 
always  indicated  a definite  agency  preference. 

Recommendations  for  Further  Research 
This  research  effort  is  considered  a basic  initial 
examination  of  a complex  subj ect--contract  modifications. 
More  research  on  this  subject  is  strongly  urged.  The 
literature  existing  at  this  point  in  time  is  unfortunately, 
sadly  lacking.  The  present  research  has  revealed  several 
areas  within  the  subject  of  contract  modifications  that 
are  recommended  for  further  study. 


:.0»  , V 
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Recommendation  One 


A study  similar  to  this  research  should  be  made 
within  a single  Federal  agency.  For  example,  the  question- 
naire could  be  administered  to  the  major  acquisition  divi- 
sions within  the  Air  Force  Systems  Command. 

Recommendation  Two 

An  analysis  of  the  responses  to  question  seven  of 
the  questionnaire  revealed  that  the  majority,  53.6  percent, 
of  contract  modifications  are  due  to  changes  in  contract 
specifications.  Research  is  needed  to  determine  why 
specifications  are  being  changed.  Is  it  due  to  factors 
such  as  new  technology,  uncertainty  in  definition  of 
specifications  t the  time  of  award,  changed  missions, 
poor  or  inaccurate  specifications,  or  other  reasons? 

Recommendation  Three 

A survey  of  defense  contractors  should  be  made  to 
gain  a perspective  of  how  they  view  existing  contract 
modification  methodology.  The  survey  should  include  the 
contractors'  perception  of  why  there  are  so  many  modifica- 
tions to  contracts. 

Recommendation  Four 

Research  is  needed  to  determine  if  any  Federal 
agencies  have  developed  any  new  contract  modification 
instruments . 
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Recommendation  Five 


The  procedures  of  managing  contract  changes  used 
by  purchasing  activities  in  the  private  commercial  sector 
should  be  studied.  These  procedures  may  be  appropriate 
for  use  in  Government  contracting. 


APPENDIX  A 

A SURVEY  OF  CONTRACT  MODIFICATION  INSTRUMENTS 


I 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  f'ORCE  INSTITUTE  OF  TECHNOLOGY  ( AU  ) 
WRIGHT-PATTERSON  AIR  FORCE  BASE.  OHIO  45433 

Af-NOF°  SLGR  (SLSR  9-77A/Capt  Sholley/Capt  Prigmore/ 
Capt  Martin/AUTCVON  78-74240) 
iuajECT-  ^ Survey  of  Contract  Modification  Instruments 
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TO: 


1.  The  attached  questionnaire  was  prepared  by  a research 
team  at  the  Air  Force  Institute  of  Technology,  Wright- 
Patterson  AFB,  Ohio.  The  purpose  of  the  questionnaire  is 
to  determine  those  contract  modification  instr’uments  pre- 
ferred by  various  Department  of  Defense  procurement  agencies. 

2.  You  are  requested  to  provide  an  answer  to  each  question. 
Headquarters  USAF  Survey  Control  Number  77-47  has  been 
assigned  to  the  questionnaire.  Your  participation  in  this 
research  is  voluntary. 


3.  Your  response  to  this  questionnaire  will  be  held  confi- 
dential. Please  remove  t.his  cover  letter  before  returning 
the  completed  questionnaire.  Your  cooperation  in  providing 
this  data  will  be  appreciated  and  will  be  very  beneficial 
in  evaluating-yaricus  contract  modification  instruments . 

''  y .'7  -7?' 


A 


HENRY  W.  PARLETT,  Colonel,  USAF 
Associate  Dean  for  Graduate 
Education 

School  of  Systems  and  Logistics 
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Quest  ion.naire 
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A SURVEY  OF  CONTRACT  MODIFICATION  INSTRUMENTS 


PLEASE  READ  THE  FOLLOWING  INSTFUCTIONS 
BEFORE  ANSWERING  THE  SURVEY 


Please  answer  all  questions.  Mark  your  answers  on  the  ques- 
tionnaire. 

When  you  have  completed  the  questionnaire  place  it  in  the 
envelope  and  return  it  to  your  instructor  at  your  next  class 
session. 

Thank  you  for  your  assistance. 


\ 

\ 
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PRIVACY  STATEMENT 


In  accordance  with  paragraph  30,  APR  12-35,  the  follow- 
ing information  is  provided  as  required  by  the  Privacy  Act 
of  1974: 


a.  Authority: 

(1)  10  U.S.C.,  80-12,  Secretary  of  the  Air  Force, 
Powers,  Duties,  Delegation  by  Compensation;  and/or 

(2)  EO  93-97,  22  Nov  43,  Numbering  System  for 
Federal  Accounts  Relating  to  Individual  Persons;  and/or 

(3)  DOD  Instruction  1100.13,  17  Apr  68,  Surveys 
of  Department  of  Defense  Personnel;  and/or 

(4)  AFR  17S-9,  9 Oct  73,  Air  Force  Military 
Survey  Program. 

b.  Principal  purposes.  The  survey  is  being  conducted 
to  collect  information  to  be  used  in  research  aimed  at 
illuminating  and  providing  inputs  to  the  solution  of  prob- 
lems of  interest  to  the  Air  Force  and/or  DoD. 

c.  Routine  Uses.  The  survey  data  will  be  converted 
to  information  for  use  in  research  of  management  related 
problems.  Results  of  the  research,  based  on  the  data  pro- 
vided, will  be  included  in  written  master's  theses  and  may 
also  be  included  in  published  articles,  reports,  or  texts. 
Distribution  of  the  results  of  the  research,  based  on  the 
survey  data,  whether  in  written  form  or  presented  orally, 
will  be  unlimited. 

d.  Participation  in  this  survey  is  entirely  voluntary. 

e.  No  adverse  action  of  any  kind  may  be  taken  against 
any  individual  who  elects  not  to  participate  in  any  or  all 
of  this  survey. 
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PAi?' 


1.  Please  indicate  your  duty  organization. 

Departzjent  of  the  Air  Force 

a.  Air  Force  Logistics  Cocurand 

b.  Air  Force  Systems  Command 

c.  Other  Major  Operating  Commands 

Department  of  the  Army 

d.  Army  Material  Command 

e.  Corps  of  Engineers 

f.  Forces  Command  or  Overseas  Command 

Depart-tent  cf  the  Navy 

g.  Naval  Material  Command 

h.  Other 


Defense  Supply  Age.ncy 

i.  Defense  Supply  Centers 

j.  Defense  Servioe  Centers 

k.  Defense  Depots 

l.  Defense  Contract  Administration  Services 


m.  Othor  Federal  Agencies 


2. 

What 

is  your  present  grade  1 

evel? 

a. 

E5  thru  E9 

d. 

GS-5  thru  GS-7 

b. 

01  t.hru  03 

e. 

GS-8  thru  GS-12 

c. 

04  thru  06 

f . 

G3-13  thru  GS-15 

3. 

How  much  total  active  federa 
I.ndicata  t.he  nearest  year  gr 

1 Cover 
oup . 

ment  ser'/ice  have 

a . 

1 thru  3 years 

d. 

11  t.hru  15  years 

b. 

4 thru  6 years 

e. 

15  t.hru  2 0 years 

c . 

7 t-hru  1 0 years 

e 

Over  20  years 

4.  aow  .-nany  years  of  federal  procurement  experience  do  you  have? 
Indicate  the  nearest  year  group. 


a.  1 through  3 years 

b.  4 t.hru  S years 

c.  7 thru  10  years 

d.  11  t.hru  15  years 


e.  IS  t.hru  20  years 

f.  Over  20  years 

g.  No  procurement  related  experience 


5.  Indicate  t.he  most  appropriate  description  of  your  prese.nt  job. 


a.  Procuring  Contracting  Officer 

b.  Administrative  Contr acti.ng  Officer 

c.  Contract  Administrator 

d.  Cost/Price  Analyst 

e.  Contract  Negotiator 

f . Buyer 

g.  Procure-ment  Staf f.'.Managemcnt 

h.  Other 
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A contract  niodiiication  is  any  written  alteration  in  the  specifications,  | 

delivery  point,  rate  of  delivery,  contract  period,  price,  quamtity,  or  ' 

other  contract  provisions  of  any  existing  contract.  Answer  the  questions 
in  Part  II  based  on  the  type  of  contract  modification  you  most  frequently 
accomplish. 

I 

S.  Please  indicate  the  price  range  of  contract  modifications  you  most 
frequently  handle. 

a.  No  cost  modifications 

b.  $10,000  or  less 

c.  Over  $10,000  t.hru  $100,000 

d.  Over  $100,000  t.hru  $1,000,000 

e.  Over  $1,000,000 


7.  What  type  of  contract  modification  do  you  most  frequently  accomplish? 


a.  Changes  in  specifications  and/or  drawings 

b.  Changes  in  time  and/or  place  of  performance 

c.  Changes  in  met.hod  of  packing  and/or  shipment 

d.  Changes  in  quantity 

e.  Changes  in  Government  Fiirnished  .“aterial 

8.  What  percentage  of  your  total  worki.ng  time  is  spent  on  contract  modifi- 
cations? 


a. 

0 % - 10% 

* 

51%  - 60% 

c. 

11%  - 20% 

g- 

61%  - 70% 

c. 

21%  - 20% 

h. 

71%  - 30% 

d- 

31%  - 40% 

i , 

81%  - 90% 

e- 

41%  - 50% 

j- 

91%  - 100% 

A contract  modification  instrument  is  a method  used  for  accomplishing  con- 
tract modifications. 

9.  Which  contract  modification  instrument  do  you  use  most  frequently? 

a.  Unilateral  change  order  issued  undor  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

o.  Unilateral  change  order  issued  under  authority  of  a contract  pro- 
vision other  t.han  the  Changes  clause. 

d.  Bilateral  supplemental  agreement  issued  under  authority  of  a con- 
tract provision  other  t.han  the  Changes  clause. 

e.  Bilateral  supplemental  agreeme- t for  performance  outside  the 
scope  of  the  contract. 

f.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  implies  a contract  is  to  be  modified. 

g.  Do  .not  know. 
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10.  Which  contract  modification  instrument  does  your  agency  prefer  you 

to  use? 

a.  Unilateral  change  order  issued  u.nder  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

c.  Unilateral  change  order  issued  ’under  authority  of  a contract 
provision  ot.her  than  the  Changes  clause. 

d.  Bilateral  supplemental  agreement  issued  ’under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

e.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 

f.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  implies  a contract  is  to  be  modified. 

g.  Do  not  know. 

11.  What  contract  modification  instrument  do  you  prefer  to  use? 

a.  Unilateral  change  order  issued  ’under  I'uthcrity  of  the  Chances 
clause. 

b.  Bilateral  supplemental  agreement  iss’ued  under  authority  of  the 
Changes  clause. 

c.  Unilateral  o.har.ge  order  issued  under  authority  of  a contract 
provision  other  than  the  Cha.nges  olause. 

d.  Bilateral  s’jpplemental  agree-T.e.nt  issued  ’under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

e.  Bilateral  supplemental  agree.ment  for  performance  outside  the 
scope  of  the  contract. 

f.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  t.he  Government  that  implies  a contract  is  to  be  modified. 

g.  Have  no  preference. 


PART  m 


The  questions  in  this  part  are  concerned  with  the  characteristics  of  con- 
tract modification  i.nstruments.  Base  your  responses  on  the  price  range 
and  type  of  contract  mcdifications  you  most  frequently  accomplis.h. 
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12.  Rank  order  the  list  of  6 contract  modification  instruments  as  to 
flexibility;  that  is,  how  useful  is  each  instrument  in  responding 
to  new  situations.  Place  the  number  1 beside  t.he  instrxsnent  you 
consider  the  most  flexible.  Place  the  number  2 beside  the  instru- 
ment you  consider  the  next  most  flexible  and  so  on  to  number  6,  for 
the  instr'tment  you  consider  the  least  flexible. 

Unilateral  change  order  issued  under  authority  of  the  Changes 

clause. 

_____  Bilateral  suppl«nental  agreement  issued  under  authority  of 
the  Changes  clause. 

Unilateral  change  order  issued  under  authority  of  a contract 

provision  other  than  the  Changes  clause. 

_____  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  than  t.he  Changes  clause. 

Bilateral  supplemental  agreement  for  performance  outside  the 

scope  of  the  contract. 

Constructive  change  order  based  on  some  informal  action  on  the 

part  of  the  Government  that  implies  a contract  is  to  be 
modified. 


13.  Rank  order  the  list  of  6 contract  modification  instruments  as  to 

manageability;  that  is,  the  instrument's  ease  of  implementation  and 
control . Place  a number  1 beside  the  instrument  you  consider  the 
most  manageable.  Place  the  number  2 beside  t.-.e  instr'ument  you 
consider  the  next  most  manageable  and  so  on  to  number  6,  for  the 
instrument  you  consider  the  least  manageable. 

Unilateral  change  order  issued  'under  authority  of  a contract 

provision  ot.her  than  the  Changes  clause. 

Bilateral  supplemental  agreement  issued  u.nder  a-uthority  of  a 

contract  provision  other  than  the  Changes  clause. 

Unilateral  change  order  issued  'under  authority  of  the  Changes 

clause. 

Bilateral  supplemental  agreement  issued  under  authority  of  the 

Changes  clause. 

Bilateral  supplemental  agreement  for  performance  outside  the 

scope  of  the  contract. 

Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Goverrunent  that  i...plies  a contract  is  to  be 
modified. 
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14.  Rank  order  the  list  of  6 contract  modification  instruments  as  to  cost 
effectiveness;  t.hat  is,  the  instrument  that  is  least  costly  to  adminis- 
ter, and  has  the  least  potential  for  contract  cost  growth.  Place  the 
number  1 beside  the  instrument  you  consider  the  most  cost  effective. 
Place  the  number  2 beside  the  Instrument  you  consider  the  next  most 
cost  effective  and  so  on  to  number  6,  for  the  instrument  you  consider 
the  least  cost  effective. 

Bilateral  suppler.e.ntal  agreement  for  performance  outside  the 

scope  of  the  contract. 

Constructive  charge  order  based  on  some  informal  action  on  the 

part  of  t.he  Government  that  implies  a contract  is  to  be 
modified. 

Onilateral  c.har.ge  order  issued  under  aut.hority  of  the  Changes 

clause. 

Bilateral  supplemental  agreement  issued  under  authority  of 

the  Changes  clause. 

Unilateral  change  order  issued  under  authority  of  a contract 

provision  other  than  the  Changes  clause. 

Bilateral  suppie-.ental  agreement  issued  under  authority  of  a 

contract  provision  other  than  the  Changes  clause. 


15.  Rank  order  the  list  of  6 contract  modification  instruments  as  to 
timeliness;  that  is,  the  Instrument  that  takes  the  least  time  to 
initiate  a.nd  complete.  Place  the  number  1 beside  the  instruument 
you  consider  the  most  timely  to  use.  Place  t.he  number  2 beside  the 
instrument  you  consider  the  next  most  timely  to  use  and  so  on  to 
number  6,  for  the  instfxtent  you  consider  t.he  least  timely  to  use. 

Constructive  change  order  based  on  some  informal  action  on 

the  part  of  the  Govsrr.ment  that  implies  a contract  is  to  be 
modified. 

Bilateral  supplemental  agreeme.it  for  performance  outside  t.he 

scope  of  the  contract. 

Bilateral  supplemental  agreement  issued  under  authority  of  a 

contract  provision  other  than  the  Changes  clause. 

Unilateral  change  order  issued  under  authority  of  the  contract 

provision  other  than  the  Changes  clause. 

Bilateral  supplemental  agreement  issued  under  authority  of  t.he 

Changes  clause. 

Unilateral  change  order  issued  under  authority  of  the  Changes 

clause. 
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APPENDIX  B 


PRETEST  SURVEY  OF  CONTRACT  MODIFICATION  INSTRUMENTS 


A SURVEY  OF  CONTRACT  MODIFICATION  INSTRUMENTS 


PLEASE  READ  THE  FOLLOWING  INSTRUCTIONS 
BEFORE  ANStVERING  THE  SURVEY 


Select  only  one  answer  to  each  question.  Answer  all  ques- 
tions . 

Mark  your  answers  on  the  questionnaire  by  circling  the 
response  chosen. 

I'Hien  you  have  completed  the  questionnaire  place  it  in  the 
envelope  and  return  it  to  your  instructor  at  your  next 
class  session. 

Thank  you  for  your  assistance. 
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1.  Please  indicate  your  duty  organization. 

Department  of  the  Air  Force 

a.  Air  Force  Logistics  Cccaiand 

b.  Air  Force  Systems  Contcand 

c.  Other  Major  Operating  Commands 

Department  of  the  Army 

d.  Army  Material  Command 

e.  Corps  of  Engineers 

f.  Forces  Command  or  Overseas  Command 

Department  of  the  Navy 

g.  Kaval  Material  Command 

h.  Other 


Defense  Supply  Agency 

i.  Defense  Supply  Centers 

j.  Defense  Service  Centers 

k.  Defense  Depots 

l.  Defense  Contract  Administration  Services 


a.  Other  Federal  Agencies 
2.  What  is  your  present  grade  level? 


3. 


a. 

E5  thru  E9 

d. 

GS-5  thru  GS-7 

b. 

01  thru  03 

e. 

GS-8  thru  GS-12 

c. 

04  thru  06 

f . 

GS-13  thru  GS-15 

How  much  total  active  federal  Government  service  have  you  ccmc 
Indicate  t.he  nearest  year  group. 

a. 

1 thru  3 years 

d. 

11  thru  15  years 

b. 

4 thru  6 years 

e. 

16  thru  20  years 

c . 

7 thru  10  years 

« 

Over  20  years 

How 

Indi 

many  years  of  federal 
cate  the  nearest  year 

procurement  experience  do  you  have? 
group. 

a. 

1 through  3 years 

e. 

16  thru  20  years 

b. 

4 thru  6 years 

f . 

Over  20  years 

c. 

d. 

7 thru  10  years 
11  thru  15  years 

g- 

No  procurement  related  expi 

5.  Indicate  t.he  most  appropriate  description  of  your  prese.nt  job. 


a.  Procuri.ng  Contracting  Officer 

b.  Administrative  Contracti.ng  Officer 

c.  Contract  Administrator 

d.  Cost/Price  Analyst 

e.  Contract  Negotiator 

f.  Buyer 

g.  Procurement  Staff /Management 

h.  Other 
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A contract  modiJication  is  any  written  alteration  in  the  specifications, 
delivery  point,  rate  of  delivery,  contract  period,  price,  quantity,  or 
other  contract  provisions  of  any  existing  contract. 

6.  Please  Indicate  the  price  range  of  contract  modifications  you  .most 
frequently  handle. 

a.  ilo  cost  modifications 

b.  $10,000  or  less 

c.  Over  $10,000  thru  $100,000 

d.  Over  $100,000  thru  $1,000,000 

e.  Over  $1,000,000 

7.  Policy  guidance  contained  in  procurement  regulations  regardi.ng  contract 
modifications  is  adequate. 

a.  Strongly  disagree 

b.  Disagree 

c.  Neither  agree  nor  disagree 

d.  Agree 

e.  Strongly  agree 

3.  What  percentage  of  your  total  working  time  is  spent  on  contract  modifi- 
cations? 


a. 

0 % - 10% 

f. 

51%  - 60% 

b. 

11%  - 20% 

9- 

61%  - 70% 

Ce 

21%  - 30% 

h. 

71%  - 80% 

d. 

31%  - 40% 

i. 

31%  - 90% 

e. 

41%  - 50% 

j. 

91%  - 100% 

\ contract  modification  i.tstrument  is  a method  used  for  accomplishing  con- 
tract modifications. 

9.  Which  contract  modification  instrument  do  you  use  most  frequently? 

a.  Dnilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

c.  Unilateral  change  order  issued  u.nder  authority  of  a contract  pro- 
vision other  than  the  Changes  clause. 

d.  Bilateral  supplemental  agreement  Issued  under  authority  of  a con- 
tract provision  other  than  the  Changes  clause. 

e.  Bilateral  supplemental  agreeisent  for  performance  outside  the 
scope  of  the  contract. 

f.  Constructive  change  order  based  on  some  i.’’.formal  action  on  t.he 
part  of  the  Government  that  implies  a contract  is  to  be  modified. 

g.  Do  not  know. 


PART  II 
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10.  Which  contract  aodif ication  instriment  does  your  agency  prefer  you  to 
use? 

a.  Unilateral  c.hange  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

c.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

d.  Bilateral  supple-tental  agreement  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

e.  Bilateral  su?ple.T.e.ntal  agreement  for  performance  outside  the 
scope  of  t.he  co.ntract. 

f.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  implies  a contract  is  to  be  modified. 

g.  Do  not  know. 

11.  '.Vhich  contract  modification  do  you  prefer  to  use? 

a.  Unilateral  chance  order  issued  u.nder  authority  of  t.he  Changes 
clause. 

b.  Bilateral  suppla-.ental  agreement  issued  u.nder  authority  of  the 
Changes  clause. 

c.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

d.  Bilateral  supplemental  agreement  issued  u.nder  authority  of  a 
contract  provision  other  than  the  Chances  clause. 

e.  Bilateral  suoplamental  acree-ment  for  performance  outside  the 
scope  of  the  contract. 

f.  Constructive  chance  order  based  on  some  informal  action  on  the 
part  of  the  Gov-srnmenc  that  implies  a contract  is  to  be  modified. 

g.  Have  no  preference. 

PABT  III 

The  questions  in  this  part  are  concerned  with  the  characteristics  of  con- 
tract modification  instruments.  For  eac.h  question,  select  one  response 
from  the  two  choices  available.  Base  your  responses  on.  the  price  range  of 
contract  modification  you  nest  fregue.ntly  handle. 

rie.Kibility ! Compare  each  of  the  contract  modification  instruments  as  to 
its  f le.xibi lity;  that  is,  how  useful  is  each  i.nstru.ment  in 
responding  to  new  situations. 

12.  Which  contract  modification  instrument  is  .most  f le-xible? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  issued  under  authcrity  of  the 
Changes  clause. 
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13.  Which  contract  nodiiication  inatr\L-aent  is  most  f leribla? 

a.  Onilateral  c.banga  order  issued  under  authority  of  the  Changes 
clause. 

b.  Cnilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

14.  Which  contract  ocdification  instrument  is  most  flexible? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  th'^n  the  Changes  clause. 

15.  Which  contract  modification  instrument  is  most  flexible? 


a.  Onilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 

16.  Which  contract  modification  instrument  is  most  flexible? 

a.  unilateral  change  order  issued  undar  authority  of  the  Changes 
clause. 

b.  Constructive  c.hange  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  implies  a contract  is  to  be  modified 


17.  Which  contract  codification  instrument  is  most  flexible? 

a.  Bilateral  supplemental  agreement  issued  under  aut-hority  of  the 
Chances  clause. 

b.  Onilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  t.he  Changes  clause. 


13.  Which  contract  modification  instrument  is  most  flexible? 


a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  t.han  the  Changes  clause. 

19.  Which  contract  modification  instrument  is  most  flexible? 


a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Cha.nges  clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 

20.  Which  contract  modification  instrument  is  most  flexible? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Ccp.structive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  t.hat  i.mplies  a contract  is  to  be  modified 
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21.  Which  contract  modification  instrument  is  most  flexible; 

a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

22.  Which  contract  modification  instrument  is  most  flexible? 

a.  Unilateral  change  order  issued  'under  authority  of  a contract 
provision  other  tha.n  the  Clnanges  clause. 

b.  Bilateral  su??le.mental  agreement  for  performance  outside  the 
scope  of  the  contract. 

23.  whic.h  contract  modification  instrument  is  .most  flexible? 

a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  tha.n  the  Changes  clause. 

b.  Constructive  cha.nge  order  based  or.  some  informal  action  on  the 
part  of  the  Governme.nt  that  i.mplies  a contract  is  to  be  .modified. 

24.  Whic.h  contract  m.odif icatio.n  i.nstrume.nt  is  most  flexible? 

a.  Bilateral  supple.me.ntal  agreement  issued  under  authority  of  a 
contract  provision  ether  tha.n  the  Changes  caause. 

b.  Bilateral  suppleme.ntal  agree-ment  for  performance  outside  the 
scope  of  the  contract. 

25.  Which  contract  modification  i.nstrume.nt  is  .most  flexible? 

a.  Bilateral  supplemental  agree.mer.t  issued  under  auohority  of  a 
contract  provision  other  tha.n  the  Changes  clause. 

b.  Co.nstructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  i.mplies  a contract  is  to  be  modified. 

26.  Which  contract  modification  i.nstr'ument  is  most  f le.xible? 

a.  Bilateral  suppleme.ntal  agreement  for  performance  outside  the 
scope  of  the  contract. 

b.  Constructive  ■'har.ge  order  based  on  some  informal  action  o.n  the 
part  of  the  Govern.me.nt  that  i-mpiies  a contract  is  to  be  modified. 


M.anace ability.  Compare  each  of  the  contract  modification  instruments 

relative  to  its  manageability;  that  is,  ease  of  imple.me.nta- 
tion  and  control"!  ’ 

27.  Which  contract  medif icatior.  instrument  is  most  manageable? 

a.  Unilateral  change  order  issued  u.nder  authority  of  the  Changes 
clau  ie. 

b.  Bilateral  supplemental  agreement  issued  under  aut.hority  of  the 
Changes  clause. 
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28.  Which  contxact  jiodif ication  instrument  is  most  manageable? 

a.  dpilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Unilateral  chainge  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

29.  Which  contract  modification  instrument  is  most  manageable? 

a.  Unilateral  cha.nge  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  issued  under  aut.hority  of  a 
contract  provision  other  than  the  Changes  clause. 


30.  Which  contract  modification  instrument  is  most  manageable? 

a.  Unilateral  change  order  issued  under  authority  of  t.he  Changes 
clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 


31.  Which  contract  modification  instrument  is  most  manageable? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause . 

b.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  i-tiplies  a contract  is  to  be  modified 


32.  Which  contract  mcdif ication  instrument  is  most  manageable? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Unilateral  c.hange  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 


33.  Which  contract  modification  instrument  is  most  manageable? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  t.han  the  Changes  clause. 


34.  Whic.h  contract  modification  instrument  is  .most  manageable? 


a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 

35.  Which  contract  modification  instrument  is  most  manageable? 

a.  Bilateral  supplemental  agreement  issued  u.nder  authority  of  t.he 
Changes  clause. 

b.  Constructive  c.hange  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  implies  a contract  is  to  be  modified 
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37. 


33. 


I-ihich  contract  tjodification  instrnnent  is  most  manageable? 

a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  ot.her  than  t.he  Chances  clause. 

Which  contract  modification  instrument  is  most  manageable? 

a.  Unilateral  cha.nge  order  issued  under  authority  of  a contract 
provisio.n  ocher  than  t.he  Changes  clause. 

b.  Bilateral  supplemental  agree-oent  for  performance  outside  the 
scope  of  the  contract. 

Which  contract  modification  i.nstrume.nt  is  most  manaceable? 


a.  Unilateral  change  order  issued  under  authority  of  a cor.trnot 
prevision  ot.her  than  the  Changes  clause. 

b.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Govern-ment  that  implies  a contract  is  to  be  modified. 

39.  Which  contract  modification  instrument  is  most  manageable? 


a.  Bilateral  supplemental  agreeme.-.t  issued  under  authority  of  a 
contract  provision  ot.her  than  t.he  Cha.nges  clause. 

b.  Bilateral  supplem.ental  agreement  for  performance  outside  the 
scope  of  the  contract. 


40. 


Which  contract  mndification  instrument 


is  mest  manageable? 


a. 

b. 


Bilateral  supple.mer.tal  agree.ment  issued  u.nder  aut.hority  of  a 
contract  provision  other  t.han  the  Changes  clause. 


Constructive  change  order  based  on  seme  informal  action  on  the 
part  of  the  Goverrjnent  that  Lmplies  a contract  is  to  be  modified. 


41.  Which 
a . 


contract  modification  instrument  is  most  manageable? 

Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  t.he  contract. 


b.  Constructive  change  order  based  on  some  informal  action  on  the 

part  of  t.he  Covernutent  that  implies  a contract  is  to  be  modified. 


Cost  Effectiveness: 


Cc.mpare  each  contract  modification  instrument  as  t 
its  cost  effectiveness;  that  is,  the  least  costly 
adnunister,  and  has  tr.e  least  potential  for  contract 
cost  growth. 


42. 


Which 

a . 

b. 


contract  modification  instr'iment  is  most  cost  effective? 

Ur...la;eral  cha.*ge  order  issued  u-nder  authority  of  the  Changes 
clause. 

Bilateral  supcle.-ental  acree.~e.nt  issued  u.nder  authority  of  the 
Changes  clause. 
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43.  Which  contract  modification  instrument  is  most  cost  effective? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

44.  Which  contract  modification  instrument  is  most  cost  effective? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  suppleniental  agreement  issued  under  authority  of  a 
contract  prevision  other  than  the  Changes  clause. 

45.  Which  contract  modification  ir.stru.ment  is  most  tost  effective? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause . 

b.  Bilateral  supplemental  agreement  for  performance  outside  the  | 

scope  of  the  contract.  i 

i 

46.  Which  contract  modification  instr-xneat  is  most  cost  effective? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Constructive  change  order  based  on  some  informal  action  on  t.he 
part  of  the  Government  that  implies  a contract  is  to  be  modified. 

47.  Which  contract  modification  instrument  is  most  cost  effective? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
C.hanges  clause. 

b.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

48.  Which  contract  modification  instrument  is  most  cost  effective? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Bilateral  supple-mental  agreement  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

49.  Which  contract  modification  instr-iment  is  most  cost  effective? 

a.  Bilateral  supplemental  agree-ment  issued  under  authority  of  the 

Changes  clause.  I 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 

50.  Which  contract  modification  i.nstrument  is  most  cost  effective? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Constructive  chance  order  based  or.  some  informal  action  on  the  i 

part  of  the  Govern.mer.t  that  implies  a contract  is  to  be  modified. 
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51.  Which  contract  lodification  instrunent  is  nost  cost  effective? 

a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  ot.her  than  the  Changes  clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

52.  Which  contract  modification  instrument  is  most  cost  effective? 

a.  Unilateral  change  order  issued  under  aut.hority  of  a contract 
provision  other  than  the  Changes  clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 


53.  Which  contract  modification  instriment  is  most  cost  effective? 

a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

b.  Constructive  change  order  based  on  sc.me  inforrial  action  on  the 
part  of  the  Gover.nmer.t  that  implies  a contract  is  to  be  modified 

54.  Which  contract  modification  instrument  is  most  cost  effective? 


a.  Bilateral  supple.mental  agree-tient  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 

vrnich  contract  modification  instrument  is  most  cost  effective? 

a.  Bilateral  supple-mental  agreem.ent  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

b.  Constructive  cha.-.ge  order  based  on  so.me  informal  action  on  t.he 
part  of  the  Covernnent  that  Lmplies  a contract  is  to  be  modified 


56.  Which  contract  modification  instr-ument  is  .most  cost  effective? 

a.  Bilateral  supplemental  agreement  for  perfcrma.-.ce  outside  the 
scope  of  the  contract. 

b.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Goverr-mer.t  t.hat  i.mplies  a contract  is  to  be  modified 


Ti.-’eliness: 


Cc.mpare  each  of  the  contract  modification  instruments  in  terms 
of  timeliness;  that  is,  takes  the  least  time  to  initiate  and 
complete. 


57.  Which  contract  modification  instrument  is  most  ti.mely? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agree-ment  issued  under  authority  of 
the  Changes  clause. 
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58.  Which  contract  acdification  instrunent  is  most  timely? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 

clause. 

b.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

59.  Which  contract  modification  instrument  is  most  timely? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

60.  Which  contract  modification  instrument  is  most  timelv? 


a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 


61.  Which  contract  modification  instrument  is  most  timely? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  chat  implies  a contract  is  to  be  modified 

52.  Whic-h  contract  modif ication  instrument  is  most  timely? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Unilateral  c.hange  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 


63.  Which  contract  modification  instr'xment  is  most  timely? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
C.hanges  clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 


64.  Which  contract  modification  instrument  is  most  timely? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  t.he 
Changes  clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 

65.  Which  contract  modification  instrument  is  most  timely? 


a.  Bilateral  supplemental  greement  issued  under  authority  of  the 
Changes  clause. 

b.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  implies  a contract  is  to  be  mcdified 
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66.  Which  contract  modification  instrument  is  most  timely? 

a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  tha.n  the  Changes  clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

67.  Which  contract  modification  instrument  is  most  timely? 

a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 

53.  Which  contract  modification  instrument  is  most  timely? 

a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

b.  Constructive  change  order  based  on  seme  informal  action  on  the 
part  of  the  Government  that  implies  a contract  is  to  be  modified. 

59.  '/fhich  contract  modification  i.nstrum.ent  is  most  tim.ely? 

a.  Bilateral  supplem.ental  agree:r'.e.nt  issued  under  authority  of  a 
contract  provision  other  t.han  the  Changes  clause. 

b.  Bilateral  supplemental  agreement  for  performance  outside  the 
scope  of  the  contract. 

70.  iiihich  contract  modificatisn  i.nstru.ment  is  most  ti.mely? 

a.  Bilateral  supp leme.-.tal  agreement  issued  under  aut.hority  of  a 
contract  prcvision^ther  than  the  Changes  clause. 

b.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Govern.T.e.nt  that  implies  a contract  is  to  be  modified. 

71.  Which  contract  modification  instrument  is  most  timely? 

3.  Bilateral  supplemental  agree-ment  for  performance  outside  the 
scope  of  the  contract. 

b.  Constructive  change  order  based  on  some  informal  action  on  the 

part  of  the  Govern.T.e.no  that  Lmplies  a contract  is  to  be  nediried. 


Overall  Efficiency;  Based  on  the  criteria  of  flexibility,  manageability, 
cost  effectiveness,  and  timeli.ness,  compare  each 
contract  modification  i.nstrum.ent  in  terms  of  overall 
ef  f icie-~.cv. 

72.  Which  contract  modification  instrument  is  most  efficient? 

a.  Unilateral  change  order  issued  under  authority  of  the  Chances 
clause. 

b.  Bilateral  supple.me.ntal  sgree-ment  issued  under  authority  of  the 
Changes  clause. 
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73. 


74. 


75. 


76. 


77. 


Which  contract  modification  instrument  is  most  efficient? 

a.  Onilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Dnilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

Which  contract  modification  instrximent  is  .most  efficient? 

a.  Unilateral  change  order  issued  under  authority  of  the  Changes 
clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  prov'ision  other  than  the  Changes  clause. 

Which  contract  modification  instrument  is  most  efficient? 

a.  Unilateral  change  order  issued  under  authority  of  t.he  Changes 
clause. 

b.  Bilateral  supplemental  agreement  foi'  performance  outside  the 
scope  of  the  contract. 

Which  contract  modification  instrument  is  most  efficient? 

a.  Unilateral  change  order  issued  under  aut.hority  of  the  Changes 
clause. 

b.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  implies  a contract  is  to  be  modified 

i-Jhich  contract  modification  instrument  is  most  efficient? 

a.  Bilateral  supplemental  agreement  issued  under  aut.hority  of  the 
Changes  clause. 

b.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 


78. 


79. 


Which  contract  modification  instrument  is  most  efficient? 

a.  Bilateral  supplemental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

Which  contract  modification  instrument  is  most  efficient? 

a.  Bilateral  supple.mental  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Bilateral  supple-mental  agreement  for  performance  outside  the 
scope  of  the  contract. 


30.  Which  contract  modification  instrument  is  most  efficient? 

a.  Bilateral  3upple.mer.tal  agreement  issued  under  authority  of  the 
Changes  clause. 

b.  Constructive  c.ba.nge  order  based  on  seme  informal  action  on  the 
part  of  the  Government  that  implies  a contract  is  to  be  modified 
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31.  Which  contract  modification  i.nstrument  is  most  efficient? 


a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

b.  Bilateral  supplemental  agreement  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 


32.  Which  contract  modification  instrument  is  most  efficient? 

a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  Changes  clause. 

b.  Bilateral  supplemental  agree-nent  for  performance  outside  the 

scope  of  the  contract.  ^ 

83.  Which  contract  modification  instrument  is  most  efficient?  j 


a.  Unilateral  change  order  issued  under  authority  of  a contract 
provision  other  than  the  C.hanges  clause. 

b.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  that  i.mplies  a contract  is  tc  be  modified. 


84. 


Which  contract  .modif ication  i.nstru.ment  is  most  efficient? 


a.  Bilateral  sap?le.mental  agree.ment  issued  under  authority  of  a 
contract  provision  other  than  the  Changes  clause. 

b.  Bilaceral  supplemental  agreement  for  performance  outside  the 
score  of  the  contract. 


35.  Whic.h  contract  modification  i.nstr'u.ment  is  most  efficient? 

a.  Bilateral  supplemental  agreement  issued  u.nder  authority  of  a 
contract  provision  other  than  t.he  Changes  clause. 

b.  Constructive  change  order  based  on  some  informal  action  on  the 
part  of  the  Government  t.hat  implies  a contract  is  to  be  modified. 

8£.  Which  contract  modification  i.-.stru.ment  is  .most  efficient? 

a.  Bilateral  supplemental  agreement  for  performance  outside  t.he 
scope  of  the  contract. 

b.  Constructive  chance  order  based  on  some  informal  action  on  t.he 
part  of  the  C-overn.m.ent  that  implies  a contract  is  to  be  modified. 
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3  5 0 ; V A R 3 6 . (1  ) D Ull  H V 


34  j?.  .'CODE  ; VAr.O  L ( ' A ' , ' 3 ' , ' C ' = I ) ( ' j ' , ' C 
3 4 5 ; ( , 'J  ' , 'K' , 'L ' = 4;  (';i'=5) 

S 4 0 F 14 .3  ><  3 E C C I E 3 ; G E C E R A L = V A ?.  0 1 TO  VA  R 1 1 

3 3 OO  FT  I J'll  5 ; 3 , 3 

b56READ  INPUT  DATA 

•35  7 S : 3 ELECTA  : 7 7A  74  /DAT.\ 


'i  0 0 ’*3  w i-  n C T 

; 

(VAROl  Eg 

L ) 

COSTREOUEEC 

I L 3 

;Ge::eral='. 

■AR02 

To 

VA  R L L 

O/'jOi  X f 

3 , 0 

07  5 -‘3  ELECT 

IP ; 

(VAROL  Eg 

1) 

0 3 0 r C-  E . . C 

IF.  .3 

^ ^ . J-  n * ? _ t 

j <i_r  L«  1 i L.  kVkV  t_i  " 1 

lAROE 

TO 

varl  L 

OC;50i'...  ^ J .t  ^ 1 

3,3 

o J G * 3 E L E C T 

IT ; 

(VAROL  e; 

3) 

b V 5 F L E w E T L C 

13  3 

; G 3 li  C L = 

•A  ROE 

T 0 

V A R 1 1 

Ij  113  ; 3 , o 

LECT  i:;(VAR01  Eo  4) 


\ 


r-T  rop  / 


PROGRAM  2 


00  Lilies  , a (iL)  : , 3,  16;  : , 1 <) 

0103 : ioe:;t; .;?i  191 , AFirsi.  haxt:;;,  ?a:c;ioR£,  suollev  class  77a 
0:CS: SELSC7: SPSS/SPSS 

ooORL'i!  ;.a;;e;co:!Tract  :;oo:ficat:o::  sur.vEf 
olovariaole  e:s7:varoi  tj  var36 
050i::put  eoiu:: ;Card 

0  60 1.; PUT  to?.::at  ;fiaED  ci>;,  i u i , i:c,  6f  i . 0,  i;c,  6F  i . 0,  ix,  SF  i . o,  i a,  6f  i . o,  ix,  u 

0 70ti  uF  CASES;19-. 

(jSOVAP.  easels  :V’AP.01,I)UTV  oaCAXIZATIUU/VAaOO.CRAOE  LEVEL/ 

090  ;VA.-.03.FEDEHAL  SERVICE  7 1;:  E/ VAR  0- , YS  S PaOC.'lT  EXF  E R IF.CC  E/ 

ioo:varo6,cuty/varou,c;;  price  ralce/varo? .type  c:i/ 
iio;vap.03,7I.;c  o;;  c::/vaR09,c.ii  v.ost  fseq  usco/ 
lCO;VARlj.C.’ll  ACESCT  P.IEFERS/VARll,C;iI  ICO  PREFERS/ 

130;  VARU,rLE.a/VAR13,FL£XS/VARli.  FLEX3/VAR15.FLEX4/ 

1 3 J ; VaR  1 6 , r L E X 5 / V AP.  1 7 , FL  EX6  / V A A 1 3 , I AC  1 / V A ?.  1 9 , .IA.<  2 / 

1  4 0 ; V A R 2 vj , 1 A 3 / V A ?.  2 1 , T A C ■*  / V A R 2 2 , X A t.  6 / V A R 2 3 , 1 AC  6 / 

146  ; V«R2-. , CJS7  1 / VaF.26  , COS  72  / VAR23  , OUST  3 / VaR2  7 , CUSEh  / 

1  60  ; VAR2  3.  COST  5 / VAR29,  COST  6/  VAR20, 7I.IE  1 / VAR31  , TI'IE  2/ 

1 jO  ; VAR  32  . TII'.e:  7VAR33, 71.1  E-./VA. 13",  7I;:E6  / VAR35,  Till  E6/ 

170;  var66,ou;i:;y 

I 73  value  labels  ; .’AROl  , ( ‘a  ' )AFLC  ( ' 3 • 'AF  3C(  'C  ' )07HER  IIAJCOIIS 
1 SO ;( ’0  • ;a;ic  1 ‘E  • icur.ps  or  ergs  ( ’f ')  Forces  of.  os  uo::;!a:;d 

1 S3  ; ( ‘G  • K.A  /T  IA7  C J!I;;a..O  ( ‘3  ' )07X£R  N A VY  ( ‘ I ' ) D S G ( ' J ‘ ) 0 S VC 

1 90  ; 1 ■ X' )3EPJ73  ( 'L  • IOCaS  ( ’-.I' )07HER  FEE  AGENCIES/ 

2 30  ; VAR02  . ( ' A • )E5  TERC  E 9 ( ' 3 "I  0 1 TIIRC  03('C')04  72RU  06 
210;('0*jG3-6  7HRU  GS-7^*E*)GS-6  ERRU  GS-12 

2  1 5 ; ( ‘F ■ ;GS -1 3 7HRC  GS-16/ 

2  2 3 ; VAR03  , ( ’A  ■ ) 1 INRC  3 YRSCS')-  7HRU  o YRSCC')7  THRU  10  YRS 
2 30  ;('0','ll  7.iRC  15  YRSCE‘)16  7:1RC  20  VRSCF'IOVER  20  YRS/ 

2-0  ; VA.Io..  , ( "A  ' ) 1 7riRC  3 YRS(‘a')4  ENRU  6 YRS(‘C‘)7  7HRU  10  YRS 
2.5;{'0‘)li  7;iRU  15  "RS('E')16  7.1RU  23  i RS  ( * F ' )0  VC  R 20  YRS 
250  ;(’G'):.U  PR0C:i7  EXPERIENCE/ 

260  ; VAR05  , ( 'A  ■ ) PCO'  *3  • )ACO  ( ‘C  ' )AO;iI.;  ( '0  ' ) A NAL  YC  7 ( ' E ' ) N EG07  lA  70R 
263 , ( T ' ) 3UYER( 'G ' )STAFF( ‘H ' JOEHER/ 

2 70  ;VAR06, ( ’A'lNO  COS7 ( ’ 3 ’ ) 5 1 OK  OR  LE3 S ( ' C ' ) 0 VE R S I OK  7HRU  SIOOK 
2 73  ; I’O'IOVER  5100K  THRU  $ 1 M ( ‘ E * )0  7ER  SI.!/ 

230  ; VARO;  . ( ‘A  • )G  PECS  ( ' 3 ' )7I:1E-PLACE  PE  R F ( ' C ' ) PAC  K -S  H I P METHODS 
2o3  ; ( ’D  • )v7Y  £ ’E  ' )CF;I/ 

2 90  ;VAR03.CA')0-lO:('3*)ll-20:{'C')21-3  07('t)')3i-4  0:('E')41-50: 
295:('F’)51-60:('G‘)61-70:('U')71-30K'I')S1-90:('J')91-100'2/ 

300  ; VAR 0 9 TO  VAR  1 1 , £ ' A ' ) U CUCC ( ‘ 3 ' ) 3 5 ACC ( ‘C  ' ) UCOOC C 0 ' J 3 S AOC 

3 1 0;  £ 'E  • )3SAOSC  £ 'F  ’ >CC0  £ 'C  ' )DO:i'T  KNOW/ 

3  20;  Va.RI  2 TO  VaR 3 5 , f 1 ) ON  E £ 2 ) TL'O  £ 3 ) 7 HR  F. E f 4 ) FOUR  £ 5 ) F I VE  ( 6 ) S I .\ / 

3  30  ;VA.R36,  £ 1 )0U;!:i7 


f’r-T 

i>L  j| 


AVniiijIh  COP'/ 


3 4 0 R i;  C 0 D i;  ; V A R 0 1 ( ' A ' , ' .i  ' , 

345;  ('I','J','K','L'  = h)( 

3 5 0 * R £ L G C T I r ; ( V A P.  G 1 £ 1 ) 

j 6 0 C A O G S T A fl  3 ; "T  A a L £ 3 = V ?.  0 6 3 Y 


= 1)  r. 

= 3 ) 


'=£)  ('C'  ,'H'  = 3) 


VAR07  3Y  VAR 09/ 
P.0  7 3’ 


3 6 5 ; V A P.  j b 3 'i  VA  P 0 7 3 V VA  P\  10/  V A AGO  3 Y 
7'30?.;:ad  ir;puT  data 
7105:  3ELGCTA : 7 7A  74  /DATA 
7 i 0 ■'  G £ L G C T I F ; ( V 7i  P 01  £ (<  2 ) 

7 3 0 C P O S 3 r A 3 3 ; T A .0  L E S = V.l  A G 6 3 Y VA  R 0 7 3 Y VA  R 0 9 / 
740;VaR06  3Y  VAR07  3Y  VAP19/VAR06  3Y  VAR07  3Y 
730-‘S£LECT  IF;(VAP01  Zq  3) 

760CPUSSTAGS  ;TA3L£3  = VA.T06  3Y  VAP0  7 3 Y VA.R09/ 

7 70  ;Va:<C6  3Y  VAP0  7 3 Y VAP 1G/VAP06  3Y  VA.T0  7 3 Y 


/ j u ! 


.GCT  IF;(VA. 


1 


4) 


7903RU33TA33  ;TA3L£5=VAP06  3Y  VAP07  3Y  VA.109/ 
o 0 0 ; \ R G 6 a 1 V 4-v  iT  0 / a i v A P.  1 -0  / V A a 0 o a Y V.-\  P u / 3 i 
o 1 0 * G E L £ C T IF;  ( V A P 0 1 £ a 3 ) 

32GC:^OS3TA3S  ;T.\3LES=Va:;0'j  3Y  VAPG7  3Y  VARG9/ 


odO;'. ivP.uo  3. 
a *4  ij  G : . J a 3 r 3 ; 
J5''/;VARGb  .1' 
V3  5F  £ .13  : 

J 4 :J  ii  ; £ j ^ , 


L .V 
V ^ fv  • J / 


7 ■ 

L i = 


■;a7.  10 /'.a.  .to: 


V O J 


I I \ J / 


£ 1 3/VA 


7A..'.j7  3 
V.\  P0  9 / 
VA  P C 7 3 ■ 


VAP  1 1 

VA  .3  1 1 

VA  P 1 1 

VA  a 1 1 

VA.T  1 1 
VA  P 1 1 


PROGRAM  3 


0 0 I V 1/  j , P ( b L ) c , 3 , 1 'j  ) ; , 10 

0 10$:  1 0 0 .'1 T : P 1 1 9 1 , AF  I r S L A Ai<T  I ^ , 1’  11 1 0‘1 0 ?.  F , S .i U LLE  Y C L.i  3 3 7 7 A 

030$ : SELECr : bFSS/SFSS 

OjORu::  ::a;ic ; actual  coutaact  ;iud  data 

0 4 0 V A A L A 3 L L L 1 5 T j V A IV  j 1 TO  V *A  .1 0 h 

0 3 0 X . . ? 0 T A L.  0 1 o 1 1 j C A ?v  D 

CGOI  ..i’UT  F OP..; AT  ; F IaFj  (1  X,  IF  i.  0,  IM,  IF  1 . 0,  1 \ , IFi  . 0,  IX,  IF  1 . 0) 

~J  ij  L C ."V  ■.>  L.  b , 7 b 

OoOV.A::  LA_>ELbjvAP0I,.Avj^l,Ci/VAPo3,C.iT  jS'FD/ 

090  ; VAPj3  , C;i  PPICE  A AX  GE  / VA  AO  4 , T Y PE  ClI 

100V.'..LGZ  LAO  ELS  ; VAPO  1 , ( 1 ) AI3  FO.vCE  (3  ) AT  1 Y ( 3 ) X A VY  ( 4 ) 0 S A / 

1 1 0 ; YAPC  3 ,(  1 ) ECOCC  ( 3 ) 1.5  ACC  ( 3 ) LCOoC  ( 4 ) 3SAOC  ( 5 ) bS  AOSC/ 

1 30;  VAF.03  , ( 1 )XO  CObT(3)310P  OP  LES  3 ( 3 ) 0 VE  P ? 1 Oi:  -$100X 

i 30  ; (4  )0  VI?.$  I OOX-bM  (5  ) 0 VE?.  S 1 "J 

l.*0;YAP04,(l)OPEC3(3)TI  U-FLACE  ?ETF  ( 3 ) PACE-5  XI  P EE  F POD  3 
130;  C4)'^TY(3)0r;i 

1 7Ori.O0EEXC  : ES  ;G  EX  ETAL=VAAO  1 To  VAP04 

lOOOt  . , .j,  j 

1 9 0 P r.  A 0 I X P . 7 9 :4  T A 

3005  : 3ELECTA:  7 7a74/T:;AX0ATA 

3ID^‘3ELEC7  IF;(VaFC1  Eg  1) 

3 30CTc;37 \3b  ;7A3LEb=VAT  03  3Y  V.aT04  OY  VAP03 
333---0  ELECT  ir;(VAT0  1 £0  1) 

3 3 0 F .’.3  9 0 E X C I E .S  ; G £ X E .P  .\  1.  = V A 0 3 
3 0 P - 0 X b ; 

350*3  ELECT  IF;  (VAR  7 1 •,  3/ 

3 3 o C ' J j b - A .j  3 ; . .A  3 L b b = . \ 3 ; 

3 3 5 * 3 E L T C T I F ; ( V -A  r.  b 1 E 0 3 ) 

3 7 C T P.  E o U E X C I E b ; G E X E R A L = V .'1.  E.  3 3 
330OPIIJX3 ; 3, o 


: o XA  R 0 4 

,’A,104  AY  VAP03 
: 0 v.\  .P  0 4 


90*SELECT  IF;(VX\R01 

To  4 ) 

0 0 C .1 0 5 j . A .'i  3 ; i A 3 L.  E b = 

TA.-.  0 3 

3 Y 

V A . V 04  0 \ 

VAR  0 3 

0 5*3  ELECT  :,^;(VAP01 

EA  4) 

1 0 F E E 0 L E X C I F.  b ; G E X E .-1. 

L - V A .% 

03 

TO  VAR  04 

3 0 P T 1 0 X 3 ; 3 , 3 

3 5 C .1 0 j b 1 A b b ; T 3 L b b = 

.\  F.  ',7  3 

0 Y' 

VA  R 0 4 3 Y 

VA  ROE 

3 0 r I . . I b u 

3 4 0$:  ..  X 3 JOE 


146 


/> 

/. 


PROGRAM  4 


010 

01. IE 

L 3 

10 

;;  MOT  (2 

5) 

V t»  1 

L(2 

5) 

020 

CALL 

A 

TTACH (17," 

7 7 

A 7 4 / D A T 

A ; " . 

1,0,,) 

030 

DO  1 

00 

I 

= 1 , 200 

0 0 

READ ( 1 

7. 

1000,  e:.d 

= 999  ) ( 

::VAL(J  ) 

, J=1 , 25) 

C50 

1 000 

roR 

:IAT(16R, 

4(611, 

1-0 

,11) 

0 6 0 

DO  50 

I I 

= 1 , 25 

0 70 

5 0 ;i 

TO 

7 ( 

1 1 ) = M 0 T 

(I 

I )+:; 

VA  L 

(II) 

000 

1 00 

CO 

'«  "" 

I.JUE 

0^0 

999 

t*  Zk 

IT 

E(6,  1001)  ( 

MOT 

(J ) 

, J = 1 

,T5) 

100 

1 301 

Fur 

.;AT(o  (15 

. 1 

a/)  ) 

no 

STOl’ 

120 

L *»  D 

147 


1 


f 


y 


MAIN  DATA  FILE 


101 

E *\  A f C A A Her 

431256 

142356 

623413 

526413 

1 

102 

1 1 E ('  C E A 3 A 3 5 A 

615234 

1 32  456 

132456 

132456 

1 

103 

FElc.ARAAuB2 

2 1 4365 

314265 

132465 

132465 

1 

104 

A E C A t-  A . 'v  A A G A 

132456 

132465 

1 3 2 64  5 

132465 

1 

105 

ODEECSDBA3A 

132465 

132465 

123465 

124365 

1 

106 

IEFFA3A5DDA 

124356 

124356 

123456 

123456 

1 

107 

DECCEBOIDDC 

623145 

631245 

541263 

231456 

1 

106 

3 EAAECAEGOC 

126534 

513246 

524136 

253461 

1 

109 

CAEEARAC  EAC 

142563 

251463 

615324 

142563 

1 

110 

3 D S A F E A G C 

231 564 

356124 

425136 

201453 

1 

1 1 1 

'.IEDDA3DA0C3 

524136 

415236 

532146 

431265 

1 

1 12 

3 EFEHDAAEGG 

423156 

524136 

324156 

423165 

1 

113 

tlEFBECAGODG 

126345 

461325 

541326 

562413 

1 

1 1 4 

AEF0KJ3 A3DD 

432156 

142356 

132465 

132456 

1 

1 1 5 

D EAA03  3 30U 0 

4352  1 6 

2 1 5346 

415326 

1 2 5 3 4 6 

1 

1 1 6 

ilEEEECAROGA 

654321 

142356 

123456 

123-56 

1 

1 1 7 

3B0BEEA1B33 

415236 

52-136 

415236 

132456 

1 

1 1 b 

D E A A ti  3 A £ 3 3 A 

253416 

425136 

415236 

162543 

1 

1 1 9 

• li<Cbo2ACD33 

231456 

425316 

513426 

214365 

1 

120 

L 0 C a 3 3 5 B 3 D 3 

132456 

I 3 2 4 6 5 

142635 

132456 

1 

121 

3F.AAEG  BIDED 

243561 

4 1 5 2 3 6 

314256 

243561 

1 

122 

(1E33ECAA3B3 

142503 

413256 

315426 

162543 

1 

123 

DDAAii3nE3GG 

432561 

523416 

234516 

165234 

\ 

124 

B 3 C C E 3 A G is  E b 

1 4 2 5 6 3 

5 2 6 1 3 4 

520314 

321546 

1 

125 

C B c.  .j  A R A A D j D 

243561 

524136 

524136 

24350  1 

1 

126 

G L.  C C 1 i 3 3 ^ o 

1 3 2 -t  5 6 

132456 

2 4 5 6 1 3 

033142 

1 

127 

u D A A r A A < V 3 3 u 

635124 

336124 

524136 

132456 

1 

12b 

11  LDbiiCA;\ooG 

425361 

526143 

1 4 2 5 36 

63512- 

1 

129 

f ! G 0 u 2 3 A 3 3 

243613 

4 1 2 >)  5 3 

2 5 3 4 6 I 

361254 

1 

1 30 

i i E D A 2 j C A 3 Pi  B 

463152 

3 51  - 62 

234561 

4 2 3 1 3 6 

1 

131 

FEFCF33DB3D 

452156 

214356 

1356-2 

341263 

1 

132 

:iDDDFBDADl)F 

342  j6  1 

342561 

3 4 2 5 6 1 

342561 

1 

133 

ED9CCCAHAAA 

132456 

132456 

231340 

132456 

1 

1 34 

A ^ C A E 3 3 A A 2k 

132-56 

- 2 3 1 5 6 

.'1-256 

132450 

1 

135 

AEDAEIiARBBE 

132456 

423156 

2 5 1 4 3 6 

241536 

1 

A 

136 

CAFFGAAAGGG 

1 63542 

142563 

1 42  563 

1 42  50  3 

1 

137 

GDCCGCAA333 

132456 

415236 

435126 

1 3 2-56 

t 

13b 

DEAAFCDADGG 

534126 

231 -56 

132450 

132456 

1 

139 

11DCCCEAC333 

615342 

516423 

5 1 5 4 2 3 

564123 

1 

140 

aedoebgaqgd 

563412 

416253 

-35612 

514362 

1 

141 

AEFFECAADCC 

423156 

324156 

163542 

265143 

1 

142 

I EEEFCDAODD 

142356 

142365 

524136 

142365 

1 

143 

DEAAiiC30R3a 

425316 

-25316 

132-56 

1 3 2 4 5 6 

1 

144 

FFFFACAAEAA 

156324 

6 3 15  2 4 

135426 

15632- 

1 

145 

vjlG.\ECEJB3b 

4 1 5 2 3 6 

524 1 36 

5236-1 

6 3 2 5 1 4 

1 

1 ^6 

LEC^CBBaCCCD 

52  1 36 

231456 

3 2 1 4 3 6 

4 _ 1 5 j o 

1 

149 


r . 


i47 

CBAAAAAAOEIC 

142560 

435216 

415236 

245316 

1 

1-4  0 

23AAECAC333 

132456 

423156 

132456 

132456 

1 

149 

u l.03cDDBGCvj 

6-:.5312 

132456 

132436 

1 32-56 

1 

15  0 

314256 

413256 

415236 

516234 

1 

151 

3SCCGACAGG3 

243561 

241355 

123456 

342561 

1 

150 

f:CAACDAC333 

314256 

423156 

213456 

1 23456 

1 

153 

AE33i:3QADDC 

132465 

132465 

132465 

1 32465 

1 

154 

B3CAACADB33 

415236 

524136 

615234 

132456 

1 

155 

DE35E A 5BDDD 

524136 

425136 

524136 

162543 

1 

15  6 

GEJCDOADDDG 

514236 

324150 

243165 

215364 

1 

157 

AEAAESABEBB 

254631 

425136 

415320 

-35216 

1 

153 

A E C C C 3 A C 3 3 G 

415263 

615254 

6 1 5 2 3 4 

251643 

1 

1 5 9 

Gwi-CEOAJAx^o 

314265 

314256 

415236 

314256 

1 

160 

Gc D0A3A3A3C 

425316 

53421 6 

142356 

142563 

1 

161 

LEAAC3  3ADD'J 

564312 

314256 

645312 

132456 

1 

162 

G E E A o C A u 3 3 l* 

132456 

415236 

415236 

132456 

1 

16  3 

A 1,  F A L A C A D D D 

534126 

415236 

1 3 2 4 3 6 

132456 

1 

164 

J E G A C D 3 H C G E 

251436 

136245 

541206 

241560 

1 

1 6 5 

3 D A A C <J  0 Li  E C 

6h521  0 

132564 

132465 

1 3 2 4 5 6 

1 

16  6 

C .1.  M ")  G 3 A L G o 

524136 

-15236 

514236 

142560 

1 

1 6 7 

;■  L.  E A G A D A b 0 

132456 

415236 

524136 

241305 

1 

1 -1  j 

J r F r 3 C A A C A G 

536421 

231-56 

425316 

132456 

1 

1 6 9 

il  ^ C C/  A 3 3 i'V  D 3 3 

4 2 5 1 3 6 

415236 

4 1 52  3 6 

631425 

1 

1 70 

A E C C E A A 4*\  A G o 

2 4 3 5 6 1 

413256 

415236 

243561 

1 

1 7 1 

E E J E L -n  r.  JJ  D C 

132456 

4231 56 

523146 

253461 

1 

1 72 

D 3 /\  A ; C 3 A E C F 

326431 

412365 

365-12 

345012 

1 

1 73 

3 3 A A n ij  A D 3 0 J 

62513- 

426153 

1 32  45  6 

201465 

1 

1 7- 

^EFAFliDACCA 

342561 

3425ol 

256134 

1 6 4 3 5 2 

1 

1 73 

u 3 iV  A r i C 3 3 J U 3 

4261 35 

134256 

3 2 5 1 4 6 

142363 

1 

1 7’j 

3 3 3 3 3 U 3 3 3 3 

12-356 

341256 

2 1 3-56 

1 2 3456 

1 

1 77 

A E 3 3 F 3 A A 3 3 2 

132-56 

3 1 ‘♦2  56 

314256 

645321 

1 

1 7 j 

3 3 U F C S c 0 0 G 

4 2 5 1 3 6 

-25136 

324156 

132456 

1 

1 7':> 

C EE3ACAA3CG 

241355 

632-15 

516234 

631245 

1 

130 

I u J . \ C A 3 E A A 

213456 

123456 

123-50 

1 3 2 4 5 6 

1 

131 

3 OF  Er  33BA33 

241356 

- 1 5 2 30 

526104 
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